Skip to content

Generic Update Runbook

The update flow for a platform version in one page: prerequisites, the order of the steps, and a checklist. It also describes the older single-pipeline layout, in which one pipeline takes the environment, region and layer as run parameters. For repositories with the current entry pipelines (hub, spoke, infrastructure, container), the detailed procedure is What to do when a new version is released.

Prerequisites

Before you start, make sure you have:

  • a work item that names the target version, planned in a sprint or change window;
  • the target version available in the code repository (tag or release branch) that the customer repository points to (Release state today);
  • access to the configuration repository (Azure DevOps or GitHub) and to the Azure tenant in the portal;
  • elevated access (PIM) if a guide asks for portal changes, such as removing policy assignments or initiatives;
  • a window announced to the workload teams (Maintenance calendar).

The flow

flowchart LR
    a["1. Current version"] --> b["2. Upgrade guides"] --> c["3. Permissions"] --> d["4. Branch + guide steps + bump"]
    d --> e["5. Canary"] --> f["6. Prod"] --> g["7. Merge + validate"]
  1. Check the current version. Open the entry pipeline of the configuration repository (main.yaml in the older layout, or every entry pipeline in the current layout), find the code version it points to, and write it down. If the customer uses GitHub instead of Azure DevOps, do the same in the workflow files that replace main.yaml.
  2. Review the upgrade guides. Start with the guide for the version just after the current version, and continue until the target version. Follow all steps of each guide in order. See the platform upgrade guides. If a version has no guide, nothing special is needed for it.
  3. Request permissions. Ask for PIM elevation (Owner, Contributor or another role) if the guides need it, and wait for approval before you change anything.
  4. Prepare the change. Create a branch, for example users/<initials>/update-to-v<version>. Do the configuration repository steps and the portal clean-up from the guides. Then change the version reference:
    • Azure DevOps: in pipelines/main.yaml, the repository resource of the code repository: set ref: to refs/tags/v<version> (or the tag form your file already uses).
    • GitHub: in .github/workflows/main.yaml, set the ref: of the checkout step of the code repository, and change the @<version> at the end of both reusable workflow references (canary and prod) to the new version.
  5. Run canary. Run the pipeline for canary, layer by layer (see below). Read each plan.
  6. Run prod. Repeat with the same guide steps. Use the validated canary configuration. Make prod-specific changes only where the guide says so.
  7. Merge and validate. Create a pull request, review it and merge to main. Run the full deployment once more from main and check that it plans no changes. Confirm the version in the repository, and that canary and prod are healthy. Record follow-up actions and exceptions on the work item.

Layer order in the single-pipeline layout

The older pipeline asks for three run parameters:

Parameter Value for the update
Environment Canary first, then Prod
Region The primary region of the platform (repeat for every other region you deploy)
Layer One layer per run, in the order below
Order Layer Note
1 foundation
2 connectivity
3 ghrunners Only when the customer uses GitHub-hosted runners
4 identity
5 management
6 firewall
7 postdeployment

The older runbooks differ on where ghrunners goes (before or after connectivity). The order above is the one in What to do when a new version is released. Check the order against the layer dependencies of your version.

For each run, review the plan before you approve. If the plan has destroys or replacements, find out whether they are expected (the guide or the release notes should mention them). If you are not sure, ask the platform's product team before you apply.

When all layers are done for canary and prod and the branch is merged, run the pipeline once more from main with environment all, region all and layer all, and check for unexpected changes.

Current layout

The current hub pipeline runs all layers in one run, in a fixed order, and plans every layer. See Deployment layers and Pipelines and workflows. The ghrunners layer is not part of the current hub pipeline.

Notes

  • Release-specific guides are cumulative when you jump several versions.
  • If a guide asks for portal clean-up, do it before the later steps.
  • Keep the configuration repository in line with what you deploy. A version pin that differs between entry files is a finding to resolve first.

Checklist

  • [ ] Work item with target version, window announced
  • [ ] Current version recorded
  • [ ] Every guide between current and target version read and done
  • [ ] Permissions arranged
  • [ ] Version reference changed everywhere it appears
  • [ ] Canary run layer by layer, plans reviewed
  • [ ] Prod run layer by layer, plans reviewed
  • [ ] Merged to main; a full run from main plans no changes
  • [ ] Result and exceptions recorded