1: Introduction

1.1. Purpose of This Document¶

This document provides the detailed implementation plan for the deployment of ArcGIS Enterprise on the Microsoft Azure cloud platform. This initiative, "The new eMap Platform," aims to establish a modernised, resilient and scalable enterprise Geographic Information System (GIS) platform.

The primary purpose of this plan is to establish a clear, actionable approach for deploying ArcGIS Enterprise in Azure, adhering to cloud-native principles and enterprise architecture best practices. This plan moves beyond treating ArcGIS as a proprietary "black box" and positions it as a mission critical but standard application server in IRD's Data and Technology Platform Roadmap.

The intent of this plan is to equip the technical teams, Cloud Infrastructure (DevOps) Engineers, Cloud Security Specialists, Data Engineers, Database Administrators and GIS Engineers, with comprehensive instructions, architectural specifications and operational guidance necessary for the successful execution of this deployment.

The plan details:

• Guiding Principles and Modernisation Approach: Core tenets such as "Azure PaaS First," "ArcGIS as a Standard Application Server," "Automation via OpenTofu and Configuration Management tool," and the strategic shift towards user-managed enterprise data will be discussed in Section 2.1 and Section 2.2.
• Phased Project Delivery: An iterative delivery model progressing through MVP, Bronze, Silver, Gold and Platinum stages to incrementally build capability and manage risk is discussed in Section 2.3.
• Data Governance Framework: Comprehensive strategies for data lifecycle management, stewardship, categorisation, storage allocation and data quality standards are discussed in Section 3.1 through Section 3.5.
• Environment Strategy and Observability: The recommended approach for isolating DEV, UAT and PROD environments alongside a robust monitoring and alerting strategy based on Azure Monitoris discussed in Section 3.6 and Section 3.7.
• Technical Architecture and Implementation: Detailed specifications for the Web, Application and Data Tiers, including Azure resource configurations, ArcGIS component setup, automation strategies, security controls, and lifecycle management across all project stages are discussed in Section 4.1 through Section 4.9.
• Migration and Decommissioning: A structured plan for migrating data, services and applications from the legacy eMap system to the new eMap platform, culminating in the formal End-of-Life (EOL) of the legacy system is discussed in Section 5.1 through Section 5.4.

Throughout this plan, emphasis is placed on leveraging Azure Platform-as-a-Service (PaaS) offerings, implementing robust automation via Infrastructure as Code (OpenTofu) and the designated Configuration Management tool and establishing clear data governance principles that maintain control of enterprise data through user-managed platforms rather than proprietary systems.

1.2. ArcGIS Enterprise and PostgreSQL Versioning Strategy¶

This Plan has been developed based on ArcGIS Enterprise version 11.4, which was the latest generally available release at the time of its drafting. However, strategic considerations regarding software versions and their support lifecycles are crucial for the long-term viability and maintainability of the new eMap platform.

ArcGIS Enterprise 11.5 is scheduled for release by Esri around June 2025. Significantly, Esri categorises its ArcGIS Enterprise releases into two main support tracks: Short-Term Support (STS) and Long-Term Support (LTS). While version 11.4 is an STS release, version 11.5 is designated as an LTS release. As an LTS release, ArcGIS Enterprise 11.5 will be the final version in the 11.x series and is anticipated to receive support until June 2029. Following this, Esri is expected to introduce the 12.x series, which will incorporate major architectural changes and whose initial releases are anticipated to be STS.

Given these factors, and to align the new eMap platform with a version offering extended support and stability, the new eMap platform should initially target the deployment of ArcGIS Enterprise 11.5. This document will be comprehensively reviewed and updated to reflect any changes, new features, or updated best practices specific to ArcGIS Enterprise 11.5 once it is officially released and its documentation becomes available.

ArcGIS Enterprise 11.5 supports PostgreSQL version 16. PostgreSQL 16 which was released in September 2023, is a currently stable and supported version and is also supported by Microsoft Azure Database for PostgreSQL and is compatible with the PostGIS extension. Consequently, the new eMap platform will initially deploy ArcGIS Enterprise 11.5 on Azure Database for PostgreSQL version 16.

PostgreSQL major versions are released annually and are supported for just over five years. PostgreSQL 16 is expected to be supported until November 2028. Considering these timelines, it is recommended that IRD initiate a project in early 2028 to plan for the upgrade of the eMap platform to the latest available Long-Term Support (LTS) version of ArcGIS Enterprise 12.x, in conjunction with migrating to its then latest supported version of PostgreSQL.

Table: ArcGIS Enterprise and PostgreSQL Support Lifecycles

1.3. Target Audience and How to Use This Plan¶

This implementation plan is specifically designed for the technical teams responsible for the deployment, configuration and ongoing maintenance of the new eMap platform.

1.3.1. Target Audience¶

The primary audience for this document comprises three key groups of technical professionals:

• Cloud Infrastructure Engineers (DevOps): This group possesses expertise in Microsoft Azure services, Infrastructure as Code (IaC) methodologies (specifically OpenTofu) and Configuration Management (CM) principles. Their core responsibilities will involve provisioning and managing the Azure infrastructure, implementing automation pipelines and ensuring the operational stability and scalability of the platform. This plan provides detailed specifications for Azure resource configuration, network architecture, automation scripts and the deployment of compute and storage resources, particularly in Section 4.x.

• Cloud Security Specialists: These specialists are responsible for ensuring the security and compliance of the new eMap platform. A strong understanding of Azure security services (e.g., Web Application Firewall, Network Security Groups, Azure Key Vault, Microsoft Entra ID, Managed Identities) and Zero Trust security principles is essential. This document will provide guidelines on implementing robust security controls, identity and access management strategies, network security configurations and data protection measures detailed throughout the plan, with particular emphasis on relevant security subsections in Section 4.x and the "Zero Trust Security Model" outlined in Section 2.1.

• GIS Engineers: This group brings extensive experience with traditional Esri ArcGIS Server environments and Microsoft SQL Server. For the new eMap project, their focus will be on developing GIS Extract, Transform, Load (ETL) processes, building geospatial workflows using VertiGIS Studio Workflow, publishing services and managing enterprise geodata. Recognising that modern cloud paradigms, DevOps practices and PostgreSQL databases may be new areas, this plan is structured around. It details the shift to cloud-native data management using Azure Database for PostgreSQL (Section 2.1.4, Section 4.4), outlines new workflow development patterns (Section 4.9) and describes the automated operational model. Sections on Data Governance (Section 3.1 through Section 3.5), and ArcGIS component configuration (Section 4.3) will be particularly pertinent.

Table: Target Audience

1.3.2. How to Use This Plan¶

This document is structured to serve as a central reference and detailed playbook for all involved teams. Team members should approach it as follows:

1. Familiarisation with Core Concepts: All team members should begin by reviewing Section 2.1: Guiding Principles & Modernisation Approach and Section 2.2: Key Modernisation Strategies. These sections establish the project's objectives, core architectural principles and the strategic rationale behind them.

2. Role-Specific Deep Dives:

   • Cloud Infrastructure Engineers will find detailed guidance in Section 4.x Technical Architecture and Implementation, particularly sections addressing infrastructure provisioning (Section 4.5: Automation & Security), environment setup (Section 3.6) and operational management (Section 4.6).
   • Cloud Security Specialists should focus on security-centric sections of the document with particular emphasis on security implementation details within Section 4.1 (Web Tier), Section 4.5 (Automation & Security) and relevant networking and identity configurations across all of Section 4.x.
   • GIS Engineers might findSection 3.x (Data Governance), Data Tier implementation details (particularly Section 4.4 focusing on Azure Database for PostgreSQL and data migration), Application Tier configurations (Section 4.3) and guidance for VertiGIS Studio Workflow development (Section 4.9) of interest.

3. Phased Implementation Guidance: The plan is organised around a phased implementation approach (Section 2.3 and reflected in the structure of Section 4.x). Teams should consult the sections relevant to the current phase of the project (MVP, Bronze, Silver, Gold, Platinum), ensuring deliverables and objectives for each stage are clearly understood and defined.

4. Cross-Functional Collaboration and Learning: Given the diverse expertise within the team and the introduction of new technologies and methodologies for some members, this document is intended to foster a shared understanding. For instance, GIS Engineers are encouraged to familiarise themselves with the automation strategies (Section 4.5) to understand the new operational paradigm, while Cloud Infrastructure and Security Engineers should develop and understanding of the data governance principles (Section 3.x) that drive many architectural decisions.

5. Best Practices and Standards: The plan incorporates Esri and Microsoft Azure best practices. It should be used as the authoritative source for approved configurations, deployment standards and operational procedures for the new eMap platform. The appendices provide supplementary information.

6. Living Document: This plan shall continue to be refined as the project progresses and new insights are gained. Updates should be managed through version control and significant changes should be communicated to stakeholders.

1.4. Project Scope: ArcGIS Enterprise Base Deployment on Azure¶

The scope of this plan is the deployment of ArcGIS Enterprise 11.4 on the Microsoft Azure cloud platform, establishing what Esri terms a "base deployment." The project scope is precisely defined to ensure a focused and methodical implementation, laying a robust foundation for future enhancements and the integration of more advanced capabilities.

The core components included within this base deployment of ArcGIS Enterprise are:

1. Portal for ArcGIS: The central web-based interface for managing and sharing geospatial content, users and services.
2. ArcGIS Server: A single ArcGIS Server federated with Portal for ArcGIS and designated as the primary "hosting server." This component is responsible for publishing web services and performing spatial analysis.
3. ArcGIS Data Store: A now mandatory part of the base ArcGIS Enterprise. In the new eMap platform, its role is strictly limited to supporting the hosting server's requirements for Portal-hosted feature layers and outputs from Portal's spatial analysis tools. It is not intended as the primary repository for enterprise geodata.
4. ArcGIS Web Adaptors: Two instances of the ArcGIS Web Adaptor will be deployed per environment:
   • One configured for Portal for ArcGIS.
   • One configured for the ArcGIS Server site. These Web Adaptors will be hosted on Azure App Service (Linux runtime with Tomcat), leveraging Azure's Platform-as-a-Service (PaaS) capabilities.

This deployment will be exclusively within the Microsoft Azure cloud platform, with a strong emphasis on utilising Azure PaaS offerings where feasible and aligned with enterprise architectural principles. The specific Azure PaaS services integral to this base deployment include:

• Azure Database for PostgreSQL: With the PostGIS extension enabled, this will serve as the user-managed Enterprise Geodatabase for authoritative, curated enterprise spatial data.
• Azure Blob Storage: Utilised for the Portal for ArcGIS content directory, map/image service tile caches (the arcgiscache directory), ArcGIS Server jobs and output directories and webgisdr backup files.
• Azure Data Lake Storage Gen2 (ADLSg2): Designated as the Raster Store, employing cloud-optimised formats.
• Azure Files: Provisioned for the ArcGIS Server config-store and system directories, and for webgisdr temporary staging.

The project encompasses the setup of three distinct environments, detailed in Section 3.6: Environment Isolation Strategy:

• Development (DEV) and User Acceptance Testing (UAT) Environments:

  • Core ArcGIS Enterprise components (Portal for ArcGIS, ArcGIS Server and ArcGIS Data Store) will be deployed on separate, dedicated Azure Virtual Machines (Ubuntu 24.04 LTS) per component within each environment. This architectural structure mirrors the Production environment to ensure consistency. Cost optimisation is achieved through lower Azure VM SKUs, reduced instance counts (e.g., single instance ArcGIS Server) and the omission of High Availability (HA) or Disaster Recovery (DR) configurations.
  • ArcGIS Web Adaptors will be hosted on Azure App Service instances configured with cost-effective App Service Plans.
  • Azure PaaS resources (e.g., Azure Database for PostgreSQL, Azure Blob Storage, Azure Files, Azure ADLS Gen2) will consist of dedicated, separate instances for each environment, provisioned with lower service tiers, reduced redundancy levels (e.g., Locally-Redundant Storage - LRS) and smaller quotas compared to Production.

• Production (PROD) Environment:

  • A multi-tier architecture will be implemented for enhanced performance, scalability and resilience.
  • Portal for ArcGIS will be deployed on dedicated Azure VMs (Ubuntu 24.04 LTS), configured for HA in later project stages.
  • ArcGIS Server will be deployed using Azure Virtual Machine Scale Sets (VMSS) on Ubuntu 24.04 LTS, enabling horizontal scalability. The initial base deployment (MVP Stage) will configure a minimum of one instance, with auto-scaling introduced in the Bronze Stage and HA features in the Silver Stage.
  • ArcGIS Data Store (Relational) will be deployed on dedicated Azure VMs (Ubuntu 24.04 LTS), configured for HA in later project stages.
  • ArcGIS Web Adaptors will be hosted on Azure App Service instances with App Service Plans appropriately sized for production loads and configured for HA.
  • Azure PaaS resources will be fully dedicated, separate, high-tier services configured with appropriate redundancy options (e.g., Zone-Redundant Storage (ZRS), Geo-Redundant Storage (GRS)), implemented progressively through project stages.

Tabe: Environment Comparison

Geographically, the Production environment will initially be deployed in the Azure Australia Southeast (Melbourne) region as part of the base deployment (MVP Stage of the Phased Implementation Plan). The architecture is designed to support future replication and disaster recovery capabilities in the Azure Australia East (Sydney) region, targeted for the Gold Stage.

Automation is a cornerstone of this project's scope (detailed in Section 4.5):

• All Azure infrastructure provisioning will be automated using OpenTofu (Infrastructure as Code).
• Virtual Machine configuration, including operating system setup (Ubuntu 24.04 LTS) and ArcGIS software installation, will be automated using the designated Configuration Management tool.
• Foundational Continuous Integration/Continuous Deployment (CI/CD) pipelines (e.g., GitHub Actions) will be established for validating and deploying IaC and CM code.

The following elements are explicitly out of scope for this base deployment:

• Advanced ArcGIS Enterprise server roles such as GeoEvent Server, Image Server (beyond the configuration of the Raster Store with ADLS Gen2), GeoAnalytics Server, Knowledge Server, or Mission Server.
• Deployment of ArcGIS Enterprise using Kubernetes.
• Utilisation of Esri's Software-as-a-Service (SaaS) offerings such as ArcGIS Online or ArcGIS Platform cloud services as core components of the new eMap platform, beyond their potential use as external data sources.

1.5. Key Objectives and Business Benefits¶

The new eMap platform is architected to be a significant advancement over legacy systems, leveraging modern cloud capabilities and adhering to stringent enterprise architectural principles.

The primary objectives of the new eMap platform are:

1. Platform Modernisation and Cloud-Native Integration:

   • Objective: Transition from traditional infrastructure paradigms to a modern, cloud-native architecture by primarily utilising Azure Platform-as-a-Service (PaaS) offerings. This includes deploying ArcGIS Web Adaptors on Azure App Service, leveraging Azure Database for PostgreSQL for enterprise geodatabases and using Azure Blob Storage, Azure Data Lake Storage Gen2 and Azure Files for specialised storage needs. ArcGIS Enterprise itself will be treated as a standard application server, aligning its management with other enterprise applications. (Corresponds to Principles 2.1.1, 2.1.2)
   • Rationale: Reduces the operational burden of managing underlying infrastructure, enhances deployment agility and ensures the platform benefits from Azure's continuous innovation in its PaaS landscape.

2. Enhanced Scalability and Performance:

   • Objective: Engineer a platform capable of dynamically scaling to meet fluctuating user demand and processing requirements. This involves employing Azure VM Scale Sets for the ArcGIS Server tier in Production, configuring Azure App Service for scalable Web Adaptor hosting and utilising appropriate performance tiers for Azure PaaS data services. (Detailed in Section 4.3.2, Section 4.1.2)
   • Rationale: Ensures consistent application responsiveness, supports future growth in data volume and user load and optimises resource consumption by scaling resources according to need.

3. Improved Resilience and Business Continuity:

   • Objective: Establish a highly available (HA) architecture within the primary Azure region (Australia Southeast - Melbourne) and a robust disaster recovery (DR) capability to a secondary Azure region (Australia East - Sydney). This is achieved through multi-Availability Zone deployments where available (Sydney) or Availability Sets (Melbourne), Azure PaaS data replication (e.g., PostgreSQL cross-region replicas, Geo-Redundant Storage) and a Global Server Load Balancer (GSLB) for automated regional failover. (Corresponds to Principle 2.1.8, detailed in Silver/Gold stages of Section 4.x)
   • Rationale: Minimises service downtime in the event of component failures or regional outages, safeguards enterprise data and ensures continuity of critical GIS services, thereby reducing business disruption.

4. Implementation of a Robust and Modern Security Posture:

   • Objective: Secure the platform using a layered defence strategy based on Zero Trust principles. This includes using a Web Application Firewall (WAF), Application Delivery Controller (ADC), Network Security Groups (NSGs) for micro-segmentation, Azure Key Vault for secrets management, Azure Managed Identities for resource authentication and integrating with the enterprise Identity Provider (IdP) via SAML/OpenID Connect for Single Sign-On. (Corresponds to Principle 2.1.7, detailed in Section 4.1, Section 4.5)
   • Rationale: Protects sensitive geospatial data and services from unauthorised access and cyber threats, ensures compliance with enterprise security policies and provides a secure environment for users and integrations.

5. Streamlined Operations through Comprehensive Automation:

   • Objective: Automate the provisioning of all Azure infrastructure using OpenTofu (Infrastructure as Code) and the configuration of Virtual Machines using the designated Configuration Management tool. Establish Continuous Integration/Continuous Deployment (CI/CD) pipelines for managing and deploying these automation scripts. (Corresponds to Principle 2.1.5, detailed in Section 4.5)
   • Rationale: Increases deployment speed and consistency, significantly reduces the potential for manual configuration errors, enhances operational efficiency by freeing up technical staff from repetitive tasks and allows for rapid, reliable environment rebuilds and modifications.

6. Optimised Cost Management and Resource Utilisation:

   • Objective: Achieve cost-efficiency through strategic use of Azure PaaS, auto-scaling mechanisms, appropriate selection of Azure service tiers (especially for DEV/UAT environments), Azure Reservations or Savings Plans for Production compute and diligent monitoring of resource consumption via Azure Cost Management. (Corresponds to Principle 2.1.6, detailed in Section 3.6, Section 4.7.2)
   • Rationale: Lowers the Total Cost of Ownership (TCO) compared to traditional on-premises deployments or less optimised cloud architectures, ensures efficient use of cloud resources and provides predictable expenditure.

7. Strengthened Data Governance and User-Managed Data Control:

   • Objective: Implement a clear data governance framework where authoritative enterprise data resides in user-managed Azure Database for PostgreSQL instances, registered as Enterprise Geodatabases. The Esri-managed ArcGIS Data Store will be limited to supporting Portal-hosted feature layers and analysis outputs. Data lifecycle management will be performed using native DBMS tools. (Corresponds to Principle 2.1.4, detailed in Section 3.1 through Section 3.5)
   • Rationale: Enhances data integrity, quality and control by empowering data stewards with direct management capabilities. It promotes open, standardised data access and simplifies integration with other enterprise data systems.

Achieving these objectives will yield significant business benefits, including:

• Increased Organisational Agility: The automated and modernised platform enables quicker deployment of new GIS capabilities and services, allowing the Department to respond more rapidly to emergencies and changing requirements.
• Reduced Operational Risk and Downtime: Enhanced resilience (HA/DR) and robust security measures directly translate to fewer service disruptions and a lower risk profile for critical GIS operations.
• Improved Decision-Making: Reliable, performant and readily accessible geospatial data and services provide a stronger foundation for data-driven decision-making across the Department.
• Lower Total Cost of Ownership (TCO): The combination of PaaS utilisation, automation and optimised resource consumption contributes to a more cost-effective GIS platform over its lifecycle.
• Enhanced Innovation Capability and Future-Readiness: A modern, scalable and standards-aligned platform simplifies integration with emerging technologies (e.g., AI/ML, Big Data analytics) and facilitates easier adoption of future ArcGIS enhancements.
• Improved Developer and User Experience: Standardised environments, automated processes and a performant platform improve the productivity and satisfaction of both GIS developers and end-users.
• Greater Control and Transparency over Enterprise Data Assets: The "User Managed" data approach, coupled with a clear governance framework, provides better visibility, control and management of valuable enterprise geospatial data.

This architecture is designed not merely as a technology upgrade but as a strategic enabler that will provide a more agile, resilient, secure and cost-effective enterprise GIS platform, capable of supporting the evolving needs of the Department and its users.