Enforcing Container Resource Requests and Limits in AKS with Azure Policy for Kubernetes

Problem: Uncontrolled container resources cause noisy‑neighbor issues and wasted spend

When developers omit resources.requests or resources.limits from pod specifications, the Kubernetes scheduler has no guidance on CPU or memory needs. This can lead to pods starving for resources, being evicted by the OOM killer, or consuming more than their fair share of node capacity, which drives up costs and reduces reliability.

Thesis: Azure Policy for Kubernetes can automatically enforce resource requests and limits across AKS namespaces

By using a built‑in Azure Policy definition that targets the Kubernetes admission webhook, you can require every new pod to declare both requests and limits. The policy works in audit, enforce, or deny modes, letting you adopt the rule gradually without breaking existing workloads.

1. Understand the policy definition and modes

The policy Require container resource requests and limits (built‑in ID: 1e30610a-9ec7-4862-90cc-61f183e70655) evaluates the spec.containers[].resources field of each pod. If the field is missing or incomplete, the policy returns a non‑compliant result.

• Audit mode – logs non‑compliant pods but allows them to run.
• Enforce mode – blocks creation of non‑compliant pods via the admission webhook.
• Deny mode – similar to enforce but returns a denial response; useful for strict lockdown.

Start in audit mode to see the impact, then move to enforce once you have updated manifests or added default values.

2. Scope the policy to your AKS cluster and namespaces

Azure Policy for Kubernetes is attached to an AKS resource group. You can scope the assignment to the entire cluster or to specific namespaces using the notScopes or scopes properties of the assignment.

Example: assign the policy to the production namespace only.

az policy assignment create \
  --name "enforce-resources-prod" \
  --policy "1e30610a-9ec7-4862-90cc-61f183e70655" \
  --scope "/subscriptions//resourceGroups//providers/Microsoft.ContainerService/managedClusters/" \
  --params "{\"effect\":{\"value\":\"Deny\"}}" \
  --not-scopes "/subscriptions//resourceGroups//providers/Microsoft.ContainerService/managedClusters//namespaces/" \
  --description "Enforce resource requests/limits in the production namespace"

Where to run: Azure CLI on a workstation with Policy Contributor or Owner role on the subscription. Replace placeholders (<subscriptionId>, <resourceGroup>, <aksClusterName>, <namespaceToExclude>) with your actual values.

3. Roll out with an audit‑first approach

Before switching to enforce, run the assignment in audit mode and review compliance.

az policy assignment create \
  --name "audit-resources-prod" \
  --policy "1e30610a-9ec7-4862-90cc-61f183e70655" \
  --scope "/subscriptions//resourceGroups//providers/Microsoft.ContainerService/managedClusters//namespaces/production" \
  --params "{\"effect\":{\"value\":\"Audit\"}}"

Check results in the Azure Portal under Policy → Compliance or via CLI:

az policy state list \
  --resource /resourceGroups//providers/Microsoft.ContainerService/managedClusters/ \
  --filter "policyAssignmentName eq 'audit-resources-prod'"

The output shows each pod that is non‑compliant (missing requests/limits). Use this list to update Helm charts, Kubernetes manifests, or add a LimitRange as a fallback.

Once the audit shows acceptable compliance (e.g., < 5% non‑compliant), change the assignment to enforce:

az policy assignment update \
  --name "audit-resources-prod" \
  --params "{\"effect\":{\"value\":\"Deny\"}}"

4. Worked example: validating a new pod

After the enforce assignment is active, try to deploy a pod without resources:

kubectl run test-no-resources --image=nginx --restart=Never

Expected result: the API server returns an error similar to:

Error from server (Forbidden): admission webhook "validation.azurepolicy.com" denied the request: [Azure Policy for Kubernetes] Pod "test-no-resources" violates policy "Require container resource requests and limits": missing resources.requests or resources.limits

Now deploy a pod that includes both fields:

kubectl run test-with-resources --image=nginx --restart=Never --overrides='{"spec":{"containers":[{"name":"nginx","image":"nginx","resources":{"requests":{"cpu":"100m","memory":"64Mi"},"limits":{"cpu":"200m","memory":"128Mi"}}}]}}'

This pod should be admitted successfully. Verify with:

kubectl get pod test-with-resources -o jsonpath='{.spec.containers[0].resources}'

You should see the requested and limited CPU/memory values.
Trade‑off and limitation
The primary limitation is that the policy only affects pod creation at admission time. Existing pods that lack resource definitions continue to run until they are recreated (e.g., via a rollout). Therefore, enforce mode can cause deployment failures if your CI/CD pipeline applies manifests without the required fields. Mitigate this by:

Running an audit pass first and updating all manifests.
Adding a namespace‑level LimitRange as a safety net for containers that omit resources.
Using CI pipeline checks (e.g., kubeval or conftest) to fail builds early.

Another consideration is performance: the admission webhook adds a small latency to each pod creation request. In most AKS clusters this latency is negligible (< 10 ms), but you can monitor API server latency via Azure Monitor if you run at very high scale.
Actionable closing

Identify the built‑in policy ID for container resource requests and limits.
Create an audit‑mode assignment targeting the AKS cluster (or specific namespaces).
Review compliance results and update any manifests or Helm charts that are missing resources.
Switch the assignment to enforce (or deny) mode once audit shows acceptable compliance.
Validate the enforcement by attempting to deploy a pod without resources and confirming the admission denial.
Periodically re‑run audit checks to catch regressions, especially after library or base image updates.

By following these steps you move from relying on developer discipline to a automated guardrail that protects cluster stability and helps control costs.