Skip to content

Release, Documentation and Delivery

How a new version of the library is made, how the documentation stays correct, and how the modules reach the platform that uses them.

Release process

A change goes through three steps: the pull request, the release on main, and the delivery to the consumers.

flowchart TD
    pr["Pull request"] --> a1["CI: documentation generated and committed to the branch"]
    pr --> a2["CI: tests of the changed modules (all tests for other changes)"]
    a1 --> m["Squash merge to main"]
    a2 --> m
    m --> b["Release pipeline: new tags per changed module"]
    b --> c1["Publish: sync the main branch and tags to the read-only repository of the consumer"]
    b --> c2["Publish: Bicep modules to the registry of the consumer"]
    w["Weekly CI run on main: all tests"] -.-> m
  1. Pull request. The CI pipeline of the technology generates the module documentation and commits it to the branch, and runs the tests of the modules that the pull request changes (Testing). The author updates the changelog and, when needed, MAJOR or MINOR in version.json (Versioning).
  2. Merge. The change reaches main as a squash commit, so that the release pipeline can read it as one change.
  3. Release. The release pipeline runs on main. It creates a tag for each module whose files changed, with the next PATCH number (Pipelines and scripts).
  4. Delivery. The publish pipeline sends the new tags to the consumers (below).

The release pipeline itself does not run tests. A full test of all modules runs every week on main, so that a failing module shows up outside of a pull request as well.

Documentation

There are two kinds, and they are produced differently:

Kind Content How
Solution and process documentation How the library works, the user guide, the ambassador guide, processes Written by hand, in the docs/ folder, in Markdown
Module documentation The parameters and outputs of each module, with an example Generated from the code, in the folder of the module

The generator depends on the technology:

  • Terraform: terraform-docs, a widely used tool. It combines the hand-written _header.md of the module (features, limitations, notes), the defaults example, and the generated requirements, resources, inputs and outputs.
  • Bicep: a PowerShell script (Set-Module) that builds the main.json of the module and writes the README with the notes, resource types, parameters, outputs and the modules it references.

The main README also holds a table of all modules, with a link per technology. A script regenerates it.

The CI pipeline runs the generators on every branch except main, and commits the changes to the branch of the pull request (commit message Updated documentation [skip ci]). The documentation is therefore correct when it is merged. Do not edit generated files by hand. The main place for all documentation is the repository itself. Everything is Markdown.

flowchart TD
    subgraph manual["Manual"]
        a["Process and solution documentation, _header.md of a module"]
    end
    subgraph auto["Automated"]
        b1["PowerShell generator for Bicep"]
        b2["terraform-docs for Terraform"]
        b["Module documentation"]
        b1 --> b
        b2 --> b
    end
    a --> d["Markdown in the repository"]
    b --> d

Pipelines

The pipelines of the library automate testing, documentation, release and delivery. The definitions are in the pipelines/ folder. Pipelines and scripts describes each one, with its trigger and parameters. In short:

Pipeline Purpose
Manual deploy and destroy Runs apply or destroy for one path of the repository from a private build agent. Use it to build and try a template.
CI for Bicep and for Terraform Generates the module documentation, creates the shared resources, runs the integration tests and the compliance check. Removes the test resources afterwards, except the shared resources.
Release Creates the new tags per module, based on the changes since the previous commit. It runs only from main, and changes must arrive as squash commits (Versioning).
Publish Delivers the library to the consumers: synchronises the repository with their read-only copy, and pushes the Bicep modules to their registry.

The publish pipeline is kept in a separate repository in the reference design, so that the list of consumers and their connection details are not visible to the other consumers. The repository contains the script that publishes a Bicep module to a registry (Pipelines and scripts).

Delivery to a consumer

A consumer gets the library in two ways:

What How
The Terraform modules, the examples and the documentation A read-only copy of the repository in the consumer's own environment. The main branch and all tags are synchronised on every release. The consumer references a module by tag (Use Managed Templates).
The Bicep modules Published as artifacts to a container registry in the consumer's tenant at every release, per module and version.

Onboard a consumer

  1. The consumer completes the setup on its side: the repository, the registry (for Bicep), a service principal for the synchronisation, and the access groups (Set up access). The consumer sends the details of the service principal, the subscription and the repository.
  2. Create a service connection in the pipeline project of the library for the consumer. Use workload identity federation, so that no secret is exchanged. Name it after the consumer.
  3. In the consumer's tenant, add a federated credential to the service principal that the connection uses: scenario Other issuer, with the issuer and the subject identifier from the service connection, a name, and a description of what it is used for. The consumer can do this, or the platform team on their behalf.
  4. Add the consumer to the list of consumers that the publish pipeline synchronises, with a pull request.
  5. Run the publish pipeline from main.
  6. Check the log, and check that the branch and the tags appeared in the consumer's repository, and that the Bicep modules are in the registry.

Offboard a consumer

  1. Remove the consumer from the list of consumers of the publish pipeline. From then on, nothing is delivered any more.
  2. Optionally remove the service connection of the consumer.