4.5: Automation & Security

This section outlines the strategy for Automation & Security across all stages of the new eMap platform; detailing the evolution of automation practices and security controls from MVP through to the Gold stage.

4.5.1 Automation & Security MVP¶

The MVP stage establishes the foundational automation and security posture for the new eMap platform across DEV, UAT and the initial PROD environments.

Automation MVP¶

The MVP for automation focuses on codifying infrastructure, standardising configurations and establishing initial CI/CD workflows.

• Infrastructure as Code (IaC) with OpenTofu:

  • Reusable Modules: Development of initial, reusable OpenTofu modules for core Azure resources. This includes Virtual Networks (VNets), subnets, Network Security Groups (NSGs), Virtual Machines (VMs for Portal for ArcGIS, ArcGIS Data Store and the initial single ArcGIS Server instance for DEV/UAT, transitioning to VMSS for PROD Server later), Azure App Service Plans and App Services (for Web Adaptors), Azure Storage Accounts (Blob, Files, ADLS Gen2), Azure Database for PostgreSQL and Azure Key Vault.
  • Environment Parameterisation: Utilisation of environment-specific .tfvars files to manage distinct configurations (e.g., SKUs, instance counts, naming conventions) for DEV, UAT and PROD environments, ensuring a single, version-controlled IaC codebase.
  • Secure State Management: The OpenTofu state file should be stored securely in an Azure Blob Storage container, configured with state locking to prevent concurrent modifications and ensure integrity. Access to this state file should be strictly controlled.

• Configuration Management (CM) Tool:

  • Initial CM scripts should be developed for:
    • Operating System Hardening: Applying baseline security configurations to Ubuntu 24.04 LTS VMs.
    • Prerequisite Installation: Automating the installation of required OS packages (e.g., PostgreSQL client libraries for ArcGIS Server VMs).
    • User and Group Management: Creating dedicated non-root service accounts for ArcGIS software.
    • System Limits Configuration: Setting and persisting file handle (nofile to 65,535) and process limits (nproc to 25,059) for the ArcGIS service accounts.
    • ArcGIS Software Installation: Silent installation and baseline configuration of Portal for ArcGIS, ArcGIS Server and ArcGIS Data Store (Relational type).

• Continuous Integration/Continuous Deployment (CI/CD) Pipeline:

  • An initial CI/CD pipeline (e.g., using GitHub Actions) should be established to automate the deployment workflow. This pipeline should include stages for:
    • Linting and Validation: Static analysis of OpenTofu code and CM scripts.
    • Planning: Generating an execution plan for OpenTofu (tofu plan) to review proposed infrastructure changes.
    • Applying: Execution of tofu apply to provision or update Azure infrastructure.
    • CM Execution: Triggering the Configuration Management tool to apply configurations to newly provisioned or updated VMs.
    • Web Adaptor Deployment: Automating the deployment of ArcGIS Web Adaptor .war files to the respective Azure App Service instances.

The following diagram illustrates the conceptual CI/CD pipeline for the MVP stage:

graph TD
    subgraph "📦 Git Repository"
        A[("📥 Code:<br>
        IaC (OpenTofu) <br>
        CM Scripts <br>
        WAR files")]
    end

    subgraph "⚙️ GitHub Actions"
        B(("🎯 Trigger: <br>
        Push/PR to main branch"))
        B --> C{"✅ Validate & Plan IaC"}
        C -- Plan Approved --> D["🏗️ Apply IaC:
        Provision/Update
        Azure Resources"]
        D --> E["⚙️ Execute CM Scripts:
        Configure VMs &
        ArcGIS Software"]
        E --> F["🚀 Deploy Web Adaptor
        .war to App Services"]
        F --> G(("🏁 Deployment Complete"))
        C -- Plan Issues --> H(("❌ Halt:
        Review Plan"))
    end

    subgraph "🧪 Post-Deployment"
        G --> I["🔍 Smoke Tests"]
        I --> J{"🧪 Tests Pass?"}
        J -- Yes --> K(("✅ MVP Environment
        Ready"))
        J -- No --> L(("🔄 Rollback/
        Remediate"))
    end

    A --> B

    classDef repo fill:#f0f5ff,stroke:#2563eb,stroke-width:2px;
    classDef pipeline fill:#f0fdf4,stroke:#16a34a,stroke-width:2px;
    classDef steps fill:#fff,stroke:#475569,stroke-width:1px;
    classDef decision fill:#ffedd5,stroke:#f59e0b,stroke-width:1px;
    classDef post fill:#faf5ff,stroke:#9333ea,stroke-width:2px;

    class A repo;
    class B,C,D,E,F,G,H pipeline;
    class I,J,K,L post;
    class C decision;
    class D,E,F,I steps;

Diagram: Conceptual CI/CD Pipeline for MVP Deployment.

Security MVP¶

The MVP security focuses on establishing essential controls aligned with the Zero Trust model.

• Network Security:

  • Network Security Groups (NSGs): Applied to subnets for each tier (Web, Application, Data) to enforce micro-segmentation. Rules should be defined to allow only necessary communication protocols and ports between tiers, based on the principle of least privilege. Default-deny rules should be in place.
  • Web Application Firewall (WAF) & Application Delivery Controller (ADC): Basic WAF policies (e.g., OWASP Top 10) and ADC path-based routing rules should be configured to protect web-facing applications and direct traffic appropriately. Administrative interfaces of ArcGIS components should be blocked at the WAF/ADC.

• Data Security:

  • Encryption at Rest: Azure Storage Service Encryption (SSE) should be enabled by default for Azure Blob, Files and ADLS Gen2. Transparent Data Encryption (TDE) can be configured for Azure Database for PostgreSQL instances if encryption at rest is required.
  • Encryption in Transit: HTTPS/TLS should be enforced for all external communication to the platform, managed at the ADC. Internal communication between Azure services (e.g., App Services to VMs) should also leverage TLS.

• Identity and Access Management (IDAM):

  • Azure Key Vault: Used as the central, secure store for all secrets, including Portal for ArcGIS and ArcGIS Server Primary Site Administrator (PSA) credentials, database passwords, API keys and SSL/TLS certificates.
  • Managed Identities: System-assigned Managed Identities should be configured for Azure VMs and App Services to enable secure authentication to Azure Key Vault for retrieving secrets and to other Azure services (e.g., Azure Storage) where appropriate, without needing to store credentials in application code or configuration files.
  • Least Privilege: Initial RBAC roles will be assigned to Managed Identities and service principals, granting only necessary permissions.

• ArcGIS Security Configuration:

  • Administrative access via ArcGIS Server Web Adaptor will be disabled.

4.5.2 Automation & Security Bronze¶

The Bronze stage builds upon the MVP foundation, enhancing automation for scaled environments and refining security configurations.

Automation Bronze¶

Focus shifts to managing dynamically scaling components, particularly the ArcGIS Server VMSS.

• CI/CD Enhancements for Scaled Environments:

  • Pipelines should be updated to manage the deployment and update of "golden" VM images for the ArcGIS Server VMSS. This includes strategies for rolling updates to minimise service disruption.
  • IaC (OpenTofu) scripts should be updated to define and manage auto-scaling configurations for the VMSS and Azure App Service Plans.

• Configuration Management for Scaled Instances:

  • CM scripts must be robust and idempotent to handle the dynamic nature of VMSS instances.
  • Automated Site Joining: New ArcGIS Server VMSS instances must automatically join the existing ArcGIS Server site. Credentials for site joining should be retrieved securely from Azure Key Vault by the new instance using its Managed Identity, orchestrated via VMSS Custom Script Extensions or CM client bootstrapping.
  • CM scripts must ensure consistent application of configurations, including system limits and service settings, to all instances. Modularity and version control for all CM scripts are critical.

• CI/CD Pipeline Maturity:

  • Pipeline triggers should be refined (e.g., on commits to specific branches).
  • Introduce approval gates in the CI/CD pipeline for deployments to the PROD environment to ensure an additional layer of review before production changes.

Security Bronze¶

Security practices are reinforced, particularly around automated processes.

• Secure Secret Handling in CI/CD:

  • The CI/CD pipeline's service principal (or Managed Identity if using self-hosted runners on Azure) should be granted least-privilege access to Azure Key Vault to retrieve secrets required during the deployment process (e.g., for authenticating to Azure, configuring resources).
  • No secrets should be stored directly in pipeline variables or logs.

• CM Script Security:

  • Ensure CM scripts handle secrets securely, retrieving them from Key Vault at runtime via Managed Identities on the target VMs.
  • Environment-specific configurations (licences, settings) applied by CM scripts should be managed using environment variables or CM attributes, avoiding hardcoding sensitive information.

• Vulnerability Management:

  • Establish processes for regular vulnerability scanning of "golden" VM images and deployed VM instances.
  • Integrate security scanning tools into the CI/CD pipeline.

4.5.3 Automation & Security Silver¶

The Silver stage focuses on automating the deployment and configuration of High Availability (HA) features within the PROD (Melbourne) environment.

Automation Silver¶

Automation scripts are extended to manage HA configurations.

• IaC and CM for HA:

  • OpenTofu modules and CM scripts should be updated to support the deployment and configuration of HA pairs/clusters for Portal for ArcGIS (two-machine site), ArcGIS Data Store (primary-standby) and the ADC.
  • Parameterisation should be key to differentiate between non-HA DEV/UAT environments and the HA-enabled PROD environment, allowing the same core automation codebase to manage all configurations.
  • CM scripts should handle the specific clustering and replication setup required by ArcGIS components for HA.

• VMSS Custom Script Extensions: VMSS Custom Script Extensions will continue to play a role in bootstrapping CM clients or running initial configuration scripts on new ArcGIS Server instances, ensuring they correctly join the site and are configured for HA operations.

Security Silver¶

Security measures are adapted to support HA configurations.

• Secure Communication in HA Clusters: Ensure secure communication channels (e.g., encrypted, authenticated) between nodes in HA clusters (Portal, Data Store, ADC). NSG rules will be reviewed and updated to permit necessary inter-node traffic for HA synchronisation and heartbeats, while maintaining the principle of least privilege.
• Key Vault for HA Secrets: Any new secrets or shared keys required for HA configurations (e.g., for cluster communication, shared storage access by HA pairs) should be stored securely in Azure Key Vault, accessed by the relevant components using Managed Identities.
• Access Control for HA Management Interfaces: Access to any management interfaces specific to HA configurations (e.g., Portal HA admin tools, Data Store replication status) should be appropriately restricted.

4.5.4 Automation & Security Gold¶

The Gold stage extends automation and security to support inter-Region Disaster Recovery (DR) for the PROD environment (Melbourne to Sydney).

Automation Gold¶

Automation plays a critical role in provisioning the DR environment and orchestrating failover/failback.

• IaC for DR Environment:

  • OpenTofu scripts should be used to provision the "pilot light" infrastructure in the Sydney DR region. This includes minimal instances of WAF, ADC, App Services, Portal VM(s) (which might be stopped or use smaller SKUs to save costs during normal operations), ArcGIS Server VMSS (configured with a minimum of 1 instance, ready to scale on demand post-failover) and ArcGIS Data Store VM(s) (also potentially stopped or minimal).
  • The IaC code will define the target state for the DR environment, allowing for rapid scaling and activation during a DR event.

• DR Runbooks and Automation Scripts:

  • Detailed DR runbooks should be developed, outlining procedures for failover and failback.
  • Automation scripts should be created to automate key DR procedures, such as:
    • Promoting the Azure Database for PostgreSQL read replica in Sydney to a writable primary.
    • Initiating storage account failover for GRS accounts.
    • Scaling up "pilot light" compute resources (VMSS, App Services) in Sydney.
    • Restoring the ArcGIS Enterprise application state using webgisdr backups from geo-replicated storage.
    • Updating GSLB configurations to redirect traffic to Sydney.

Security Gold¶

Security practices are extended to the DR environment and processes.

• Secure DR Environment Provisioning:
  • The DR environment in Sydney should be provisioned with the same security rigour as the primary Melbourne environment. This includes applying NSGs for tier isolation, enforcing HTTPS/TLS and using Azure Key Vault for secrets management specific to the DR site if needed (though a central Key Vault with regional replication would be preferred).
• Secure Data Replication:
  • Ensure all data replication mechanisms (Azure Database for PostgreSQL cross-region replicas, Azure Storage GRS) are configured securely, with encryption in transit for replication traffic.
• Key Vault Access in DR:
  • Ensure Managed Identities of resources in the DR region have the necessary permissions to access relevant secrets in Azure Key Vault.
• GSLB Security:
  • Secure health probes from the GSLB to regional endpoints.
  • Protect the GSLB configuration from unauthorised changes.
• Esri Security Best Practices: Consistently apply Esri security recommendations in both primary and DR environments.
• Database Credentials: Database connection credentials (e.g., for ArcGIS Server to connect to Azure PostgreSQL) must be stored securely in Azure Key Vault and retrieved at runtime. Direct embedding of credentials or reliance on OS-authenticated .sde files for service connections should be prohibited. Limit database user permissions.

By systematically implementing and evolving automation and security controls across these phased stages, the new eMap platform will achieve a high degree of operational efficiency, deployment reliability and a robust security posture.