Secret Watcher: Secret Expiry Monitoring

An expired secret is a common cause of a failing pipeline or application. The secret watcher is a scheduled check that looks at the secrets of the app registrations the platform depends on, and raises an alert before they expire. This page describes the pattern and what to configure. The recurring task around it is in the maintenance calendar.

What it watches

Item	Examples
Deployment identities of the platform	The service principals in mcp.json that the pipelines use (Operating responsibilities)
Spoke service principals	One per spoke (Change spoke)
Other app registrations the platform team owns	Anything listed in the mapping below

The check reads the client secrets (password credentials) of each listed app registration. In the version described here it does not read certificates or federated credentials. Federated credentials have no secret to expire, which is one reason to prefer them (App registration secrets).

How it works

flowchart LR
    s["Schedule<br/>(weekly)"] --> p["Pipeline loops<br/>over tenants"]
    p --> sc["Script"]
    sc -- "read-only Graph access" --> e["Entra ID<br/>app registrations"]
    sc -- "alert" --> o["On-call / incident tool"]

1. A pipeline runs on a schedule. In the example it runs weekly, on Monday morning.
2. The pipeline loops over all tenants in scope and runs the script once per tenant.
3. The script signs in to Microsoft Graph as a reader service principal in that tenant (client credentials).
4. It looks up the app registrations listed for the tenant in a mapping file, and reads their secrets.
5. For each secret it calculates the remaining time and sends an alert to the on-call tool if the secret is close to expiry.

Thresholds in the example

Remaining time	Alert severity	Message
30 days or more	None
Less than 30 days	Info	The secret will expire within 30 days
Less than 3 days	Critical	Urgent: the secret expires in <days, hours>
Already expired	Critical	Urgent: the secret has already expired

Choose your own thresholds. They must give the owner enough time to rotate, including the time to request approval and run the pipeline. A weekly check with a 30-day first warning gives about four chances to act.

What you need

Item	Notes
A reader service principal per tenant	Permission to read app registrations (the Microsoft Graph application permission Application.Read.All). Give it no more than that
A mapping file	Per tenant: the tenant ID, a name, and the list of app (client) IDs to watch. Example below
A secure store for the credentials	For example a pipeline variable group or a key vault. The pipeline reads the client ID, secret and tenant ID per tenant
An integration key of the on-call tool	The alert is sent as an event to the tool (see Roll out monitoring)
A pipeline with a schedule	Loops over the tenants and calls the script

Example of the mapping file, with placeholders:

{
  "customers": [
    {
      "tenant_id": "<tenant-id>",
      "name": "<organisation name>",
      "spn_ids": [
        "<app-id-1>",
        "<app-id-2>"
      ]
    }
  ]
}

The reader service principal has a secret too

The identity that watches the secrets also has a secret that expires. Put it on the same list, and rotate it on a fixed schedule. If you cannot get permission to read the app registrations in a tenant, you cannot watch that tenant: put the secrets of that tenant on a manual list in the monthly health check.

Add or change a tenant

Step	Add a tenant	Change a tenant
1	Create the reader service principal in the tenant and give it read access to app registrations	Update the entry in the mapping file (new or removed app IDs)
2	Add the entry to the mapping file	Check that the credentials in the secure store are current
3	Store the credentials of the reader in the secure store	Check that the script still processes the entry
4	Add the tenant to the list of tenants in the pipeline, with its credentials	Check the pipeline parameters
5	Run the pipeline once and check that the tenant is read and that a test secret near expiry would alert

When an alert arrives

1. Find the owner of the app registration. For platform identities this is the platform team. For an application registration that belongs to a workload, it is the workload team (Maintenance calendar).
2. Rotate the secret, or replace it with a federated credential where the service supports it.
3. For a platform deployment identity, use the rotation procedure of the platform (the update-spn pipeline stores a new secret in the hub Key Vault, see Pipelines and workflows).
4. Register the rotation in the change record and check the next run of the pipeline that uses the secret.
5. Remove registrations nobody owns after a notice period.

Limits

• It watches only the app registrations in the mapping file. A new app registration is invisible until someone adds it.
• It reads client secrets only, not certificates (for example Application Gateway certificates in Key Vault).
• It is a check, not a rotation. Someone still has to act on the alert.

Related pages

• Maintenance calendar
• Monthly health check
• App registration secrets