Chapter 10Publishing user-managed data

Objectives

• Understand data storage options.
• Review data store items.
• Understand the benefits of sharing data by reference.
• Create a data store item.

Introduction

In this chapter, we turn our attention to user-managed data in ArcGIS Enterprise. Unlike ArcGIS-managed data, which resides in ArcGIS Data Store and is fully controlled by ArcGIS, user-managed data remains in external storage, such as folders, databases, or even cloud storage. Organizations adopt this approach when they need fine-grained control over database management, security, and performance. This approach is also adopted when advanced capabilities such as versioning, attribute rules, and utility networks are used in workflows.

• Versioning allows multiple users to edit data simultaneously without locking the data.
• Attribute rules enforce data integrity through constraints and calculations.
• Utility networks model and analyze complex infrastructure systems.

These capabilities need to be enabled at the database level.

User-managed data

User-managed data is data that is shared by reference to ArcGIS Enterprise. In this data storage strategy, the underlying data resides outside the immediate purview of the ArcGIS Enterprise portal. Instead of copying that data to ArcGIS Enterprise during publishing, administrators choose to store, manage, and maintain it in their own infrastructure or data systems, such as a relational database. This publishing method is beneficial to larger or more specialized organizations that rely on a centralized data strategy. If a team has optimized an enterprise geodatabase or maintains a file-based repository, they can continue using their existing setup. ArcGIS Enterprise connects to this external data to support the published web layers, allowing organizations to retain their established data management practices.

For many organizations, these user-managed data sources are not new: They may already be home to critical assets, project-based archives, or authoritative enterprise datasets that feed into business workflows. By configuring ArcGIS Enterprise to reference these sources directly, teams can continue to apply familiar workflows, such as database indexing, daily backups, and existing security rules. This approach stands in contrast to simply placing data under the control of ArcGIS Enterprise, which can be more convenient for certain scenarios but forfeits some of the administrative depth and advanced workflows that managers may require.

For example, a city planning department uses ArcGIS Enterprise to host its zoning and land-use data. By hosting the data, the city can scale its system to support numerous web layers without worrying about complex database management. This setup allows planners to access and update zoning information in real time using ArcGIS Field Maps, ensuring that everyone has the latest data without needing direct database connections.

In contrast, for example, a transportation agency has an extensive enterprise geodatabase containing critical infrastructure data, including road networks and traffic signals, in which editors need to make concurrent edits in an isolated environment. By configuring ArcGIS Enterprise to reference this geodatabase directly, the agency can continue using its established workflows, including multiuser editing with versioning.

Benefits of publishing data by reference

Multiuser editing and dynamic updates

One of the most significant advantages of user-managed data is the ability to accommodate multiuser editing in an isolated environment without locking or duplicating the data. Because the information remains in a relational database or other shared environment, multiple editors can simultaneously apply updates, whether they are editing attributes, geometry, or running batch processes that transform the data each night. This leads naturally to the second benefit, which is dynamic updates. Any changes that occur in the external database automatically appear when a user accesses the web layer, so there is little or no delay when it reflects the latest edits. This is because the web layer is dynamically referencing the underlying geodatabase.

For example, a county government published its parcel data by reference to ArcGIS Enterprise with the purpose of using branch versioning. By using branch versioning, each editor can create their own named version without affecting each other’s changes. Once all edits are finalized, the changes are verified during the conflict review process. The vetted edits are then reconciled and posted to the default version so that the most accurate and up-to-date information is available to all users and the underlaying database.

Data integrity and support for complex data models

The user-managed strategy also places data integrity at the forefront. Database administrators can adopt robust measures to ensure that relationships, constraints, and rules remain consistent across all layers and tables. If advanced workflows are needed, an externally managed enterprise geodatabase is often the best environment for the sake of performance and scalability. An example of an advanced workflow is the utility network model, which requires specialized relationships among assets, such as lines, transformers, or valves.

Organizations can extend the concept of data integrity into intricate modeling of utility systems, attribute rules, topologies, or archiving for historical analysis of changes over time.

For example, a city park department wants to automate the data entry process and cut down on user errors. For that, the city publishes its data by reference to ArcGIS Enterprise to make use of on-demand attribute rules. The city creates a batch calculation rule to calculate the total number of planned trees in each park. A validation rule is also deployed to automatically flag trees in poor condition. This automated approach saves time and reduces the potential for human error, ensuring that the department’s data is accurate and up-to-date.

Data management

Because the source data is stored in a database managed by the organization, the data can seamlessly be incorporated into broader data management strategies. For instance, the same database might also serve data to applications unrelated to GIS, such as business intelligence tools or asset management systems. This integration helps teams avoid silos, ensuring that their authoritative data does not end up duplicated or scattered across multiple environments. Furthermore, once a reference web layer is deleted, the associated server is automatically deleted, but the source data located in the enterprise is retained.

For example, a water utility integrates its GIS data with broader data management strategies by storing it in an enterprise geodatabase. This database serves not only GIS applications but also business intelligence tools and asset management systems. For instance, the same database provides data for hydraulic modeling, outage management, and regulatory compliance reporting. This integration prevents data silos and ensures that authoritative data is not duplicated or scattered across multiple environments. When a reference web layer is deleted, the source data in the database is retained and can be further used for hydraulic modeling.

Types of data store items

To share a web layer by reference to ArcGIS Enterprise, the external data storage location must first be registered as a data store. More specifically, the data store needs to be registered with the ArcGIS Server site that will be used to support the web service. This may be the hosting server, or it may be another federated ArcGIS Server site. Registration is the process by which ArcGIS Server is informed about how to connect to an external data source. On a foundational level, a few common storage types can be used in a user-managed scenario.

One of the most common types of user-managed stores is the enterprise geodatabase. These databases are set up in relational database management systems, such as Oracle, Microsoft SQL Server, PostgreSQL, or other supported platforms. Within the geodatabase, organizations may implement versioning, role-based security, sophisticated indexing, or topological relationships for geometry validations.

Alternatively, some smaller teams may rely on folders containing file geodatabases or mobile geodatabases for simpler local editing.

ArcGIS Server can also connect to cloud stores if they have been appropriately registered, which might involve referencing Amazon S3 or similar storage services that hold large volumes of geospatial files. Additionally, organizations can use cloud data warehouses, such as Google BigQuery, Snowflake, or Amazon Redshift, to store and manage vast amounts of geospatial data, providing scalable and flexible solutions for data-intensive applications.

Web layers that can reference a registered data store

When a data store is registered with ArcGIS Server, ArcGIS Enterprise can create several types of web layers, each serving a different set of needs. These references ensure that ArcGIS Enterprise only points to the external data rather than copying it, preserving the advanced capabilities and structure of the original environment.

• A feature layer delivers geometry and attribute information to clients as discrete features, which can be queried, displayed, and edited if permissions allow. For example, city planners might rely on a feature layer that references an enterprise geodatabase storing parcel boundaries. Because the layer references the external database, any updates performed directly on the geodatabase become visible in the application as soon as they are committed.
• Scene layers are specialized for 3D contexts, allowing the rendering of multipatch geometry, 3D buildings, or point cloud data in a web scene. When scene layers reference an external data store, large or complex 3D datasets remain in their optimized location and benefit from any back-end infrastructure that has been established for high-volume storage.
• Imagery layers expose large raster or mosaic datasets in a dynamic or cached fashion. By pointing to user-managed data, ArcGIS Server can rely on that external source for performance optimizations, advanced image parameters, and immediate updates if new images are swapped into the mosaic. This approach is often critical for organizations, such as environmental agencies or agricultural companies, which generate or process new imagery regularly.
• Map image layers dynamically render map content on the server side, supporting advanced cartographic output or labeling logic that can be crucial for professional-grade mapping. Through user-managed data, organizations gain the benefit of full control over symbology, labeling rules, and data transformations while letting ArcGIS Server handle final rendering.
• Vector tile layers deliver vector-based data in a tiled format. Although some teams prefer to generate hosted vector tile packages using ArcGIS Data Store, many large organizations already maintain massive point, line, and polygon datasets in external databases and simply want to tile them in place for efficient consumption. By registering their data store, they can produce vector tiles from the user-managed source, giving them the performance benefits of a tiled approach and the flexibility of an external environment.

By understanding the various ways that ArcGIS Enterprise can point to external data sources, organizations can tailor their publishing approach to match each project’s needs. Some workflows might favor a feature layer for straightforward editing, whereas others might use scene layers for immersive 3D visualization. In each scenario, the key principle remains the same: The data is never copied or hosted but instead references the external data store. This ensures that the organization retains elevated control over security policies, update schedules, and advanced geodatabase functionalities, all while harnessing the powerful capabilities of ArcGIS Enterprise for web-based access and collaboration.

Tutorial 10: Create a data store in ArcGIS Enterprise

In this tutorial, you will create a new data store item in the ArcGIS Enterprise portal. Then you will use that data store item to publish multiple web layers. In the next chapter, we will cover one of the most common approaches to publishing data to ArcGIS Enterprise using ArcGIS Pro.

Create a data store item

You will use a geodatabase to create a data store item and then publish a web layer from it. For this tutorial, you will need access to an .sde file that stores the database connection for the enterprise database you want to create the data store item for.

1. Sign in to your organization ArcGIS Enterprise portal. Ensure that your account has publishing privileges.
2. On the navigation bar, click the Content tab and then click the New item button.
3. In the New item window, scroll down and select Data store.
4. For Select the type of data store to add, select Database and then click Next.
5. For Select the type of database, select Relational database and then click Next.
6. For Specify connection or location information to allow the ArcGIS Server sites to access the data, click Select File.
7. Browse in File Explorer to the location of the .sde file that stores the database connection to the enterprise geodatabase you want to create a data store item for.
8. Select the .sde file and click Open.

   The connection information appears. Your connection information will be different, based on your server, instance, database, and other properties.

9. Click Next.
10. Check the box next to your server to select it.

    

11. Click Next.
12. For the Data store connection settings, enter the following information:
    • Title: Your data store item name.
    • Folder: An appropriate folder to save your data store in.
    • Tags: Any appropriate tags for your data store.
    • Summary: This is a new data store item created directly in the ArcGIS Enterprise portal.
13. Click Create connection.

    On the Content tab, the item is listed as a data store.

    

    Next, you will use the data store item to publish multiple web feature layers directly from the ArcGIS Enterprise portal.

Publish multiple web feature layers

In these steps, you will bulk publish web feature layers from the data store item. This will eliminate the need to manually publish each web layer one by one.

1. On the Content tab, click the data store item you previously connected to open the item page.
2. Click the Layers tab and then click Create Layers.
3. For Choose Layer Properties, change the Time zone of the data to the time zone you are located in. Then click Next.
4. For Choose folder, select the same folder you created the data store item in. Click Next.
5. For Choose server, verify that your server is selected and click Start publishing.

   Depending on how many feature classes your enterprise geodatabase has, you will see a feature layer and a map image layer for each feature class. In ArcGIS Enterprise, map image layers and feature layers are often published together because they serve complementary purposes. Map image layers are typically used for displaying data dynamically or as cached image tiles and are great for faster rendering and advanced symbology. Feature layers provide access to the underlying geographic features and their attributes, which are needed for editing and running analysis.

Summary

In this chapter, you have explored the advantages of publishing to ArcGIS Enterprise data that references a registered data store. You examined the various types of storage locations that can be registered as a data store, along with their corresponding web layer types. Additionally, through the guided tutorial, you created a data store item that references an enterprise geodatabase. You used the bulk-publishing feature in ArcGIS Enterprise to publish multiple layers simultaneously. The upcoming sections of the book will provide a more in-depth examination of all the elements discussed so far by following a structured workflow with an overarching example in the next five chapters.

Отправить