Service Bus Automation (Platform Side)¶
This page is for the platform team (the integration platform engineers). It describes the three layers of Service Bus management, how the Terraform of the automation works, what it needs, how to run it and how to troubleshoot it. Workload teams: see Service Bus.
Reference design
The variable names and defaults come from the Terraform of the automation. Your platform may run a different version or different naming patterns. Check the code you run.
The three layers¶
Service Bus management is split so that the owner of the "pipes" and the owner of the "content" can work independently (ADR-012).
| Layer | Content | Code lives in |
|---|---|---|
| 1. Shared service | The Premium namespace, its private endpoint, the managed identity for encryption keys, optional customer-managed key | The shared services code, deployed to the shared services spoke (Shared services) |
| 2. Spoke extension | Permissions of an integration spoke on the namespace, created once per spoke when the spoke is deployed | The spoke extension code (Spoke extensions) |
| 3. Service Bus automation | Queues, topics, subscriptions, rules and the access groups per entity, from JSON files | The control plane code (this page) |
Deploy in this order: layer 1, then layer 2, then layer 3. A layer validates that the layer below exists. If the layers get out of sync (for example the extension points to a deleted namespace) the automation fails.
Layer 1: shared service¶
- A Service Bus namespace on the Premium SKU. Premium is needed for private endpoints and customer-managed keys (ADR-001).
- A private endpoint in a subnet of the shared services spoke. Public network access is off, and the namespace is reachable only through the private endpoint. Local (key-based) authentication is disabled in the reference shared service.
- Microsoft-managed encryption is the default. A customer-managed key can be configured. The user-assigned managed identity for that is always deployed.
- Messaging units (capacity) are scaled manually by the platform team. In the reference code, capacity must be 1, 2, 4, 8 or 16. Scaling can reset connections for a moment, so treat it as an operational change.
- Partitioning defaults are applied centrally. Increasing partitions on a running namespace is destructive and needs a migration plan.
- Geo-disaster recovery is optional.
- A platform policy can forbid workload-owned Service Bus namespaces, which enforces centralised messaging.
Layer 2: spoke extension¶
The extension prepares an integration spoke so that its service principal can manage entities but not the rest of the namespace. The integration platform source describes it like this:
- Two custom roles per spoke: one that allows all actions on queues, topics and subscriptions only, and one that gives read access to the namespace resource only, not to its children.
- A Service Bus Namespace Readers group with the spoke's Owners, Contributors and Readers as members by default.
- A Service Bus Entity Contributors group for the spoke's Owners and Contributors, for support operations.
- Names of all groups are configurable.
Check the extension on your platform
The generic documentation of the extension lists the roles somewhat differently (built-in data sender and receiver roles on the namespace, a reader group and manager rights for the spoke deployment principal), see What each extension creates. Verify the roles of the extension version you run before you rely on either description.
Layer 3: the automation¶
The automation reads configuration files from a folder, creates the entities and the groups, and assigns the roles.
flowchart TB
cfg["configuration folder<br/>queues.json, topics.json, workload.json<br/>in any subfolder"] --> tf["Terraform root: infra/service-bus"]
tf --> mod["entities module"]
mod --> q["queues sbq-..."]
mod --> t["topics sbt-..., subscriptions sbts-..., rules"]
mod --> g["Entra ID groups<br/>(in the administrative unit)"]
g --> r["Role assignments on entities"]
g --> rd["Membership of the platform readers group"]Key properties:
- Recursive discovery (ADR-003): all files named
queues.json,topics.jsonandworkload.jsonin the folder and its subfolders are merged. The folder layout is free. - Configuration, not code (ADR-002): workload teams provide data, the Terraform enforces naming and settings.
- Group-based access (ADR-005, ADR-007): see below.
- Reprocessing queues (ADR-008): a queue per subscription.
- The automation reads the existing namespace (it does not create it) and needs its subscription, resource group and name.
Resources and naming¶
| Object | Azure name | Notes |
|---|---|---|
| Queue | sbq-<workload>-<name> |
|
| Topic | sbt-<workload>-<name> |
|
| Subscription | sbts-<subscription workload>-<subscription name> |
Forwards to the generated queue when auto_generate_queue_for_topic_subscription is true |
| Generated queue | sbq-<subscription workload>-<topic name>-<subscription name> |
Inherits size and duplicate-detection settings from the topic |
| Sender group | from integration_workload_entity_sender_group_naming |
One per queue and topic |
| Receiver group | from integration_workload_entity_receiver_group_naming |
One per queue and per non-forwarded subscription |
| Operators group | from integration_workload_operators_group_naming |
One per workload |
Groups are created in the given administrative units, are security-enabled, and have two owners: the identity that runs
the automation and the service principal of the workload. The operators group has the automation identity as owner. Roles
are assigned on the entity (Azure Service Bus Data Sender, Azure Service Bus Data Receiver). The operators group gets
both on every entity of the workload. The workload's reader group and the operators group are added to the platform
readers group, which exists from the spoke extension.
Group names are built by replacing {workload} and {entity} in the templates. A template without {entity} gives
every entity the same group name, so use both placeholders for the sender and receiver patterns. Entra ID limits group
name length, so very long workload and entity names can be a problem (ADR-007).
Variables¶
| Variable | Meaning | Default |
|---|---|---|
service_bus_subscription_id |
Subscription of the namespace | required |
service_bus_resource_group_name |
Resource group of the namespace | required |
service_bus_namespace_name |
Name of the namespace | required |
integration_platform_administrative_unit_ids |
Administrative units for the groups | required |
integration_platform_readers_group_object_id |
Group with read access to the namespace (from the extension) | required |
integration_workload_operators_group_naming |
Operators group name pattern with {workload} |
required |
integration_workload_entity_sender_group_naming |
Sender group name pattern | required |
integration_workload_entity_receiver_group_naming |
Receiver group name pattern | required |
auto_generate_queue_for_topic_subscription |
Generate a queue per subscription | true |
configuration_folder_path |
Folder with the JSON files | required |
The output groups lists, per workload, the object IDs of the sender and receiver groups. Give these to the workload
team, or let them look the groups up.
Example (use your own values):
service_bus_subscription_id = "<subscription-id>"
service_bus_resource_group_name = "<resource-group>"
service_bus_namespace_name = "<namespace>"
integration_platform_administrative_unit_ids = ["<administrative-unit-id>"]
integration_platform_readers_group_object_id = "<group-object-id>"
configuration_folder_path = "./configurations"
integration_workload_operators_group_naming = "<prefix>-{workload}-operators"
integration_workload_entity_sender_group_naming = "<prefix>-{workload}-{entity}-sender"
integration_workload_entity_receiver_group_naming = "<prefix>-{workload}-{entity}-receiver"
Run the automation¶
Prerequisites are on Prerequisites. Normally a pipeline runs these steps. You can run them by hand in a development setup.
cd infra/service-bus
terraform init
terraform validate
terraform plan -out=tfplan
terraform apply tfplan
The root configuration uses an azurerm backend with Entra ID authentication, so provide the backend settings at init.
The providers are azurerm (4.x) and azuread (3.x). The root requires Terraform 1.10.1 or later.
Read the plan:
- New queues, topics and groups from the JSON files are expected.
- No unexpected deletes. A removed entity removes its groups and the messages in it.
- Check names against the naming patterns.
After the apply, verify in the portal: the queues and topics exist on the namespace, the groups exist in Microsoft Entra ID (in the administrative unit), and the role assignments are on the entities.
Pull request review checklist¶
| Check | Why |
|---|---|
File name is exactly queues.json or topics.json, JSON is valid, name and workload present |
Otherwise the entity is silently not found |
| The workload has a workload definition | Without it the plan fails |
| No duplicate entity names across folders | Folders are merged, a duplicate is hard to see (ADR-003) |
Settings are sensible (max_delivery_count, TTL, size) |
Namespace capacity is shared |
| The plan has no unexpected deletes | Deleting an entity deletes its messages |
| New sender outside the spoke has a firewall rule request | Connections are not implicit |
Review rules for configuration changes in general: Making a change safely.
Troubleshooting¶
| Symptom | Check |
|---|---|
| Queues or topics are not created | Is configuration_folder_path right? Do the files exist (**/queues.json, case-sensitive)? Is the JSON valid? Are name and workload present? |
| Groups are not created | Is integration_platform_administrative_unit_ids set? Does the identity that runs the automation have the Groups Administrator role scoped to the administrative unit? Did the apply run? |
| Permission errors on the namespace | Is the subscription ID right? Does the identity have Owner or Contributor on the resource group of the namespace? Does it have access to the state backend? |
| An application gets "unauthorized" after it was added to a group | Wait a few minutes for the group change to propagate. Check the right group (sender or receiver, right entity). |
| The plan fails on a missing workload | Add the workload definition for every workload used in a queue, topic or subscription. |