Deployment and Environments

This page explains how the portal is built, how an environment is described and how a change reaches production. It describes the reference design: pipelines, repository layout and names may differ in your deployment. For the first deployment of an environment, follow Onboarding a customer.

Overview

flowchart LR
    pr[Pull request] --> lint[Lint and<br/>Terraform tests]
    lint --> build[Build image once<br/>no Azure credential]
    build --> r1[Ring 1<br/>plan, approve, apply]
    r1 --> r2[Ring 2]
    r2 --> r3[Ring 3]

Principle	Detail
Build once, deploy many	One container image is built per run and pushed to the registry of each environment. The build job has no Azure credential.
One Terraform root per environment	Each customer and environment has its own main.tf, so a change for one customer cannot alter another by accident.
The plan is applied unchanged	The plan that was produced and shown is the plan that is applied. It is not recalculated at apply time.
Configuration in a file	The facts of an environment are rows in one JSON file, not pipeline variables.

Terraform roots

infra/
  modules/            shared modules
  platform/ado        private connections from the provider network
  platform/pagerduty  shared escalation policy and schedule
  <customer>/<env>/   one root per environment

The shared modules:

Module	Creates
backstage	Web app with staging slot, identities, private endpoints, PostgreSQL, role assignments
postgresql	PostgreSQL Flexible Server 16
key_vault	Key Vault with RBAC, purge protection, no public access
container_registry	Premium registry, no public access
azure_monitor	Application Insights and Log Analytics
tenant_configuration	Secrets of the tenant in Key Vault
cost_insights	Data Factory, storage, Data Lake, Azure SQL (Cost insights setup)
foundry_platform	Azure AI Foundry account, project, model deployments (AI setup)
pagerduty	Service, dependency and Azure Monitor action group
landing_zone_spoke_canvas	Data-only: reads the existing spoke resource group, VNet, subnets, Key Vault and Log Analytics

A typical environment root composes: spoke canvas, naming, resource group, App Service plan, Key Vault, a secret for the MCP token, container registry, Azure Monitor, tenant configuration, backstage, foundry_platform and pagerduty. The root passes the app settings of the web app: APP_CONFIG_organization_name, feature flags, the model endpoint, the MCP token and optional integration and module registry settings. See Configuration.

Shared module sources

Some environment roots reference shared modules from a registry or repository outside this code base. The source does not describe that mechanism, so confirm with the provider where those modules come from.

The environments file

Every environment is one row in infra/customer-environments.json. A schema validates the file when a pull request is opened, so a missing field fails before merge.

Field	Meaning
owner, environment	Customer and environment name (acc, prod)
deploymentEnvironment	GitHub Environment name, <owner>_<env>
ring	Deployment order
enabled	Skip the row when false
terraform.directory	Path of the environment's root
terraform.backend	Resource group, storage account and container (default tfstate) of the state
terraform.backstageClientSecretName	Name of the repository secret with the Terraform client secret
terraform.appSecretNames	Names of optional app secrets (GitHub App)
azure.clientId, tenantId, subscriptionId	Identity and subscription used for plan and apply
azure.clientSecretName	Only for the client secret fallback
azure.useDeployEnvironmentForPlan	Attach the GitHub Environment to the plan job
backstage.containerRegistry	Registry login server
backstage.appService.name, resourceGroup	The web app

Example row (placeholders):

{
  "owner": "<customer>",
  "environment": "acc",
  "deploymentEnvironment": "<customer>_acc",
  "ring": 2,
  "terraform": {
    "directory": "infra/<customer>/acc",
    "backstageClientSecretName": "BACKSTAGE_CLIENT_SECRET_<CUSTOMER>_ACC",
    "backend": {
      "resourceGroup": "<backend-resource-group>",
      "storageAccount": "<backend-storage-account>"
    }
  },
  "azure": {
    "clientId": "<client-id>",
    "tenantId": "<tenant-id>",
    "subscriptionId": "<subscription-id>"
  },
  "backstage": {
    "containerRegistry": "<registry>.azurecr.io",
    "appService": { "name": "<web-app-name>", "resourceGroup": "<resource-group>" }
  }
}

Rollout in rings

Environments deploy in numbered rings. Ring N starts only when ring N-1 has completed as a whole: if one environment in a ring fails, the next ring does not start. This is a ring-wide safety gate and not a per-customer isolation. The ring of each environment is in the ring field, so check the file for the current assignment.

For each environment in a ring the pipeline:

1. Produces a Terraform plan and a readable summary, without a GitHub Environment by default.
2. Waits at the GitHub Environment gate.
3. Applies the saved plan.
4. Deploys the portal, if the run includes it (Slot swap).

Each environment also holds its own concurrency slot for the whole deployment, so two runs cannot change the same Terraform state at the same time. The pipeline does not compare branches or commits, so an engineer can deploy a branch or an older commit on purpose, for example for a rollback.

Pipelines

Workflow	Purpose
CI/CD (push to main)	Lint, Terraform tests, build, then ring 1, 2 and 3 with infrastructure and portal
Dispatch	Manual run. Inputs: deploy_to (ring 1, 2 or 3), run_lint, deploy_infra, build_and_deploy_backstage
Terraform plan	Plans changed roots on pull requests and each environment at the start of its ring
Deploy environment	Applies the saved plan and optionally deploys the portal
Platform private connections	Plans on pull requests, applies on merge
Platform alerting	Applies the shared escalation policy and schedule

The run has a timeout of 210 minutes, because creating a Foundry account can take more than two hours.

Environment gate

The GitHub Environment named in deploymentEnvironment carries the approval. Acceptance normally has no reviewers; production requires approval by the platform team. The plan job does not use the environment unless azure.useDeployEnvironmentForPlan is true, so that the readable plan exists before anyone approves.

Slot swap

The portal deploys through the staging slot of the web app:

1. Push the image to the registry of the environment.
2. Start the staging slot.
3. Set the container image on the slot.
4. Preview swap.
5. Smoke test: the readiness endpoint returns 200 and a response header x-build-id equals the run ID. Timeout 600 seconds.
6. On success, swap. On failure, reset the swap.
7. Stop the staging slot.

Authentication to Azure

Plan and apply authenticate with OIDC through a federated credential on an app registration in the target tenant. No Azure secret is stored in GitHub.

Federated credential	Subject
Pull requests	repo:<owner-id>/<repo-id>:pull_request
Main branch	repo:<owner-id>/<repo-id>:ref:refs/heads/main
Each environment	repo:<owner-id>/<repo-id>:environment:<owner>_<env>

All have audience api://AzureADTokenExchange. The issuer is the OIDC issuer of the GitHub Enterprise tenant that hosts the repository. This differs from the public GitHub issuer, so do not copy it from generic examples. The subject uses the immutable owner and repository IDs, so a rename or transfer does not break the credentials.

Secret	Use
BACKSTAGE_CLIENT_SECRET_<OWNER>_<ENV>	Repository secret with the client secret of the portal's app registration, passed to Terraform. It must be a repository secret because the plan runs without a GitHub Environment.
GH_APP_*	Optional GitHub App values, named in appSecretNames, exposed to Terraform as var.app_secrets and stored in Key Vault, referenced from app settings
AZURE_CLIENT_SECRET_<OWNER>_<ENV>	Only when no federated credential can be created. A last resort: a long-lived secret that you rotate by hand.

Plan and apply share one identity

The per-environment federated credential does not yet separate the plan phase from the apply phase, because both use the same identity. The decision record lists this as an open risk (Decisions).

Updating the portal

A change to the code goes through a pull request and then through the rings. A configuration change for one customer is a change to its app-config.<customer>.yaml or to the app settings in its Terraform root. See Configuration. Customer-specific code changes are not made.

Related

• Onboarding a customer
• Operations
• Operating models