Cleaning Up a Demo or Test Deployment¶
Demo, test and training environments, and failed implementation attempts, need to be removed again so they do not cost money or leave objects behind in the tenant. This page describes how to tear down a platform environment and how to build it up again from scratch.
Only for environments you can lose
A destroy removes the hub's networking, firewall, policies and management groups, and the spokes with their Key
Vaults and storage accounts. Use it only on a demo or test platform environment, or on the canary environment
of a customer who agreed to it. Never run it against a production platform environment. Check the mcp.json that
the destroy pipeline targets before you start.
Order: spokes first, then the hub¶
- Remove the spokes with the
spoke-destroypipeline, one spoke file at a time (spokeGlobset to that file). Use the samemcp.jsonas thespokerun that created them. The destroy should run the spoke environments in the reverse order of the deployment (prd, acc, tst, dev). Check that your copy of the entry file does, because older copies ran dev first (Known issues). Details and the data-loss warning: Delete spoke. The pipeline does not cancel the subscription and does not revert its tags or name. - Remove the hub with the
hub-destroypipeline. It runs the layers in reverse order: postdeployment, then firewall, identity and management, then connectivity, then foundation (Deployment layers), for each region. In the reference configuration repository it targets canary only. - Check what is left (see below).
- Remove the job runners and the state only if the whole environment goes away: the runner resource group, the Terraform state storage account, the service principals, and the Azure DevOps or GitHub objects.
Check the entry files before you run them: mode: Apply applies the destroy without an approval stop, and whatIf: true
only plans it. Run once with whatIf: true, read the destroy plan, and run again with whatIf: false
(Pipelines).
Do not cut off your own agents
If the runner virtual network is connected to the hub with internet_security_enabled = true, destroying the
connectivity layer removes the agents' route to Azure DevOps or GitHub, and the next steps cannot run. Prevent this by
connecting the runner network with internet_security_enabled = false, or run the last layers from a different
runner (see Network access of the agents).
What can remain after a destroy¶
| Remains | Why | Action |
|---|---|---|
| Subscriptions, with their tags and names | The pipeline never cancels subscriptions | Decommission by hand, or reuse them (Delete spoke) |
| Soft-deleted Key Vaults | Soft delete and purge protection keep them. A Key Vault that is recreated with the same name is recovered with its old secrets (recover_soft_deleted_key_vaults = true). |
Purge them if you want a clean start (Known issues) |
| Custom policy definitions and initiatives at the management group | Older cleanup pipelines removed the policy (set) definitions explicitly before the foundation layer was destroyed | List the custom definitions and set definitions at the platform management group and remove leftovers |
| Management groups and role assignments | A failed or partial destroy can leave them | Check the management group tree and the IAM of the platform management group |
| Entra ID groups and service principals of spokes | Created in the administrative unit by the spoke pipeline | Check that the spoke destroy removed them |
| Terraform state files | Not deleted by destroy | Delete the blobs for the removed layers and spokes if you will recreate them with different values |
| Resource groups of the job runners | Deployed by infrastructure, not by the hub |
Remove them when the whole environment goes away |
Scheduled clean-up of a demo environment¶
A demo environment that is used occasionally should be removed automatically. An older cleanup pipeline did this for the canary platform environment on a schedule (every night): for each region it destroyed the postdeployment, firewall, identity, management and connectivity layers, removed the policy (set) definitions, and destroyed the foundation layer. Its only environment-specific setting was the platform management group ID.
You can build the same with a scheduled run of hub-destroy for the demo platform environment. Keep two rules:
- Schedule it only on an environment that holds nothing worth keeping.
- Run the clean-up by hand as soon as you finish a demo or test, so you do not pay for the idle environment overnight.
Deploying from scratch again¶
For a regular customer a clean (first-time) deployment happens once. For a demo environment that you destroy regularly it
happens every time. A clean deployment needs the layers and regions in a specific order, because they depend on each other
for resources such as the virtual network: the hub pipeline handles it
(Deploy the platform). Use the full hub run, not single layers.
Before the rerun:
- [ ] The job runners and the
mcpimage still exist (or runinfrastructureandcontainerfirst) - [ ] The state storage account and the spoke state container still exist
- [ ] Leftover policy definitions and soft-deleted Key Vaults are cleaned up as above
- [ ] The subscriptions are back under the platform management group (a new management group tree is created by the foundation layer)
- [ ] The configuration still matches your target. Remove the spoke files of the spokes you destroyed, or the next spoke run recreates them.