Deployment Layers¶
The hub is split into separate Terraform root modules called layers, in infra/deploy/<layer>/. Each layer has its
own state file and its own plan and apply, and runs in a fixed order.
Why layers¶
The assessment behind the split: Architecture notes.
- Blast radius. A firewall rule change only plans the
firewallstate. It cannot accidentally touch management groups or gateways. - Dependency order. Later layers read resources that earlier layers created, such as the virtual hub, the firewall policy or the management groups.
- Speed and locking. Smaller states plan faster, and runs on different layers do not block each other's state lock.
- Different change cadence. Firewall rules and policies change weekly; gateways and the Virtual WAN rarely change.
The layers¶
| Level | Layer | Regions | Main resources | Subscription |
|---|---|---|---|---|
| 1 | foundation |
primary | Management groups and subscription placement; custom policy definitions and initiatives; resource provider registration; action groups; activity log alerts; DDoS plan (optional); Azure Virtual Network Manager and IPAM pools (optional); Private DNS policy definitions | management, connectivity, identity |
| 1 | connectivity |
primary | Virtual WAN, virtual hub, Azure Firewall and firewall policy, VPN/ExpressRoute gateways, point-to-site VPN, private DNS zones and resolver, Bastion, Application Gateway and WAF policies, alert processing rule, hub 24/7 routing (optional) | connectivity |
| 2 | foundation |
other regions | Regional parts of foundation (for example the regional action groups) | as above |
| 2 | connectivity |
other regions | Regional hub of each extra region | connectivity |
| 3 | firewall |
all | IP groups and firewall policy rule collection groups (from JSON); Network Manager security admin rules (optional) | connectivity |
| 3 | identity |
all | Identity virtual network and its virtual hub connection, alert processing rule | identity (+ connectivity) |
| 3 | management |
all | Management virtual network and hub connection; Key Vault, Automation Account, Log Analytics, storage, Sentinel (optional); alert processing rule | management (+ connectivity) |
| 4 | postdeployment |
primary | Built-in compliance initiatives; policy assignments and exemptions (from JSON); Defender for Cloud settings; AMBA alert initiatives | management |
Within level 1 and level 2, connectivity waits for the foundation apply (in Plan mode they plan side by side). Level 3 layers run in parallel; level 4 waits for all three.
Where the order is defined:
- Pipelines: as job dependencies in
.azure-devops/pipelines/templates/stage-mcp-hub.yamland asneeds:in.github/workflows/stage-mcp-hub-deploy.yaml. The region matrix per level comes fromSet-XebiaRegionMatrix.ps1. - Local tooling:
src/Xebia.Mcp/Functions/Private/Config/Get-XebiaOrderedDeployments.ps1, used byDeploy-,Remove-andUnlock-XebiaLocal.
Primary and other regions come from primary_region and active_regions in <env-folder>/global.auto.tfvars.
Several foundation and postdeployment resources exist only once per platform environment, even though the layer runs in
more regions. They are guarded with count = var.scaffold_location_short_name == var.primary_region ? 1 : 0. Examples:
management groups, policy definitions, policy assignments and alert initiatives.
Layers that are not part of the hub pipeline¶
infra/deploy/ghrunners and infra/deploy/policy_exemptions exist in the code, but neither the hub pipeline templates
nor Get-XebiaOrderedDeployments include them. Only legacy pipelines use them.
- Policy exemptions are deployed by the
postdeploymentlayer instead. - A
ghrunners.auto.tfvarsfile in a configuration repository has no effect on the current hub pipeline. (Older update runbooks still listghrunnersas a layer to run; that applies only to the legacymain.yamlpipelines.)
Destroy order¶
hub-destroy runs the same layers in reverse order (postdeployment → … → foundation): see the destroy jobs in
stage-mcp-hub.yaml and Remove-XebiaLocal, which calls Get-XebiaOrderedDeployments -Reverse.
Parallelism¶
| Layer | terraform -parallelism |
|---|---|
firewall |
1 (Azure Firewall Policy accepts one rule collection group update at a time, ADR-0009) |
postdeployment |
10 |
| all others | 100 |
Set in Invoke-XebiaHubPlanAction.ps1 and Invoke-XebiaHubApplyAction.ps1. A large firewall change therefore takes
noticeably longer.
Terraform state¶
| What | Backend (mcp.json) |
State key |
|---|---|---|
| Hub layer | Azure.TerraformState.Hub |
<env>-<region>-<layer>.tfstate, e.g. prod-gwc-firewall.tfstate |
| Spoke subscription | Azure.TerraformState.Spoke |
<workloadShortName>_<environment>_Subscription.tfstate |
| Spoke canvas | Azure.TerraformState.Spoke |
spoke_<workloadShortName>_<environment>.tfstate |
<env> is the name of the folder that contains mcp.json. Spoke keys are lower-cased. The backend uses Entra ID
authentication (use_azuread_auth=true), so the deployment identity needs a data-plane role on the state storage account.
Which change touches which layer¶
| Change | Layer(s) that show changes in the plan |
|---|---|
| Firewall rule or IP group | firewall |
| Application Gateway or WAF policy | connectivity |
| Hub feature switch (VPN, Bastion, App Gateway, DNS zones, …) | connectivity (most switches), see feature flags |
| Management group or subscription placement | foundation (primary region) |
| Custom policy definition or initiative | foundation (primary region) |
| Policy assignment or exemption, alert initiatives, Defender settings | postdeployment |
| Log Analytics, Key Vault, Automation | management |
| Spoke JSON | spoke pipeline (not a hub layer) |
The hub pipeline always plans all layers. A layer with no changes gives an empty plan and an apply that does nothing. To run a single layer, use the local tooling, see Run a single layer locally.