AKS Platform: Decisions¶
This page condenses the design decisions of the solution. Each row gives the decision, the reason and where it shows up. The sources record most of them as confirmed during design. Where the code differs from the decision, the last column says so.
Cluster and network¶
| # | Decision | Reason | Code status |
|---|---|---|---|
| 1 | Two clusters: production, and one non-production cluster with a namespace per environment | Blast radius, independent upgrades, different access rules, compliance, network separation | Implemented |
| 2 | Two spokes, no peering between them | Strong network isolation | Implemented. A third, hub-integrated cluster exists for non-production |
| 3 | Azure CNI in overlay mode | Avoids IP exhaustion on the node subnet, simple subnet planning, pod ranges can overlap between clusters | Implemented |
| 4 | Calico as the network policy engine | Mature, broad policy support. The cluster baseline chapter still says Azure Network Policy | Calico in code |
| 5 | Private API server with VNet integration | No public endpoint, no authorised IP list needed | Implemented |
| 6 | Local accounts disabled, Entra ID with Azure RBAC | One identity source, MFA, no shared admin credential | Implemented |
| 7 | Egress forced through Azure Firewall with a NAT gateway on the firewall subnet | Inspection, logging and scalable SNAT | Not deployed by the Azure root. Standalone uses the load balancer. Hub-integrated uses user-defined routing |
| 8 | Gateway API for ingress, platform owns the gateway, teams own routes | No lock-in to one controller | Manifests in the GitOps repository |
| 9 | Private endpoints for all PaaS, centrally owned by the platform team | Consistent governance | Key Vault only |
| 10 | No web application firewall in the initial deployment | Keep the first deployment simple | Not present |
| 11 | Region westeurope only, so no region in resource names |
Single region | Implemented |
| 12 | Hub-integrated mode as a second networking mode with the same output contract | Reuse the landing zone without changing the layers above | Implemented. The design document still says "implementation pending" |
Cluster operations¶
| # | Decision | Reason | Code status |
|---|---|---|---|
| 13 | Same Kubernetes minor version at steady state, non-production first | Validate before production | Process |
| 14 | Production upgrade channel none, non-production patch |
Deliberate production changes, current non-production | Implemented |
| 15 | System pool with CriticalAddonsOnly, user pool general |
Separate platform from workloads | Implemented |
| 16 | Production across three zones, non-production smaller | Availability versus cost | Non-production is smaller in code than in the design |
| 17 | Azure Monitor, image cleaner, managed Prometheus and diagnostics off by default | Cost. Diagnostics mandatory before production go-live | Off in the reference files |
| 18 | Defender for Containers disabled | Not enabled initially | Switch exists, off |
| 19 | The cluster is disposable: rebuild with Terraform and restore with GitOps | Simplicity | Persistent volume backup is not covered |
| 20 | One Terraform root and state per platform and environment, one configuration file per environment | Isolated blast radius, simple deployment | Implemented |
Identity and secrets¶
| # | Decision | Reason | Code status |
|---|---|---|---|
| 21 | Kubernetes native RBAC deployed by GitOps, Azure RBAC for Azure resources | GitOps compatible, reviewed by pull request | Implemented, Azure RBAC for Kubernetes is also switched on |
| 22 | Group naming grp-aks-<scope>-<role>[-<team>][-<env>] |
Predictable | Implemented |
| 23 | Production developers view only, acceptance restricted | Least privilege | In the access matrix |
| 24 | One workload identity per application and environment | No shared permissions | Implemented |
| 25 | GitHub Actions with federated identity for CI/CD, separate identities per duty | No secrets, separation of duties | Implemented |
| 26 | Break-glass through PIM, time-limited, monitored | Emergency access with a trail | PIM is configured manually |
| 27 | Groups and role assignments owned by the platform team in Terraform | One owner | Implemented |
| 28 | Key Vault with RBAC, private endpoint only, purge protection. Applications read it with the SDK, no CSI driver | Per-application scoping, no secrets in etcd | The CSI add-on is switched on in the reference files |
| 29 | Certificates held in Key Vault, with Key Vault rotation, instead of cert-manager | No extra controller | Not verified, cert-manager is also named in the design as a platform namespace |
Security policy¶
| # | Decision | Reason | Code status |
|---|---|---|---|
| 30 | Azure Policy built-in initiatives on day one, Kyverno when ready | Baseline without a new controller, then richer rules | Azure Policy assignments at Audit and Deny as listed in Security policy |
| 31 | Kyverno over OPA Gatekeeper | YAML, mutation and generation in one tool, policy reports | Policies exist, all at Audit in the manifests |
| 32 | Policy lifecycle: audit, then enforce in non-production after 7 days, then production after 14 days | No surprises in production | Process |
| 33 | Default-deny network policy in every application namespace | Lateral movement | Manifests in the GitOps repository |
| 34 | Time-boxed policy exceptions, 90 days at most, with security approval | No permanent holes | Process |
| 35 | In-cluster manifests move to a GitOps repository as Helm charts with a configurable label domain | Per-customer labels, environment differences as values | The Terraform repository copies are deprecated |
Azure Local¶
| # | Decision | Reason | Code status |
|---|---|---|---|
| 36 | Create the cluster with the Azure CLI, not Terraform | Cluster creation with OIDC and workload identity, and MetalLB, need the Azure CLI flow | Implemented in the bootstrap script |
| 37 | Arc extensions instead of AKS add-ons: Argo CD, Azure Policy, Arc Networking | Add-ons do not exist on Azure Local | Implemented |
| 38 | MetalLB with ARP, BGP or Both |
Load balancing on premises | Implemented |
| 39 | Reuse the shared identity and Key Vault modules | Same model on both variants | Implemented |