Enforcing Shoot Resource Quotas with Gardener’s Admission Webhook

Problem: Unintended Over‑Provisioning in Tenant Gardens

When multiple teams share a Gardener‑managed landscape, a mis‑specified Shoot can request more CPU, memory, or storage than the garden’s quota allows. Without automatic checks, the cluster may be created, consuming resources that affect other tenants and leading to unexpected costs or capacity exhaustion.

Thesis: Gardener’s Built‑In Admission Webhook Guarantees Quota Enforcement

Gardener ships an admission webhook that intercepts Shoot creation and update requests. The webhook validates the Shoot’s spec against the garden‑level ResourceQuota objects defined in the gardener namespace. If the request exceeds any quota, the webhook returns an admission denial, preventing the Shoot from being persisted. This shifts quota enforcement from manual audits to an automated, API‑level guardrail.

How the Webhook Works

The webhook runs as part of the Gardener API server extension. It implements the admission.k8s.io/v1 interface and receives an AdmissionReview object containing the Shoot object. The controller:

1. Extracts the Shoot’s spec.provider.workers (node pools) and spec.kubernetes.kubeVersion.
2. Calculates the requested resources (e.g., total CPU cores = sum of worker.cpu * worker.count).
3. Compares the totals against the ResourceQuota objects labeled gardener.cloud/quota=true in the garden namespace.
4. If any limit is exceeded, it sets allowed: false and populates status.message with a human‑readable explanation.
5. Otherwise it returns allowed: true.

Because the webhook is invoked before the Shoot is stored in etcd, no persistent object is created when a quota violation occurs.

Worked Example: Configuring and Testing the Quota Check

Assume a garden named garden‑prod has a quota limiting total CPU cores to 40 across all Shoots. The following manifest creates a Shoot that requests 50 cores (5 workers × 10 cores each).

apiVersion: core.gardener.cloud/v1beta1
kind: Shoot
metadata:
  name: over-quota-shoot
  namespace: garden-prod
spec:
  provider:
    type: aws
    workers:
      - name: worker-1
        machine:
          type: m5.large   # 2 vCPU, 8 GiB
        volume:
          size: 100
        count: 5
  kubernetes:
    version: 1.27

Apply the Shoot:

kubectl apply -f over-quota-shoot.yaml

The admission webhook intercepts the request. You should observe an error similar to:

Error from server (Forbidden): admission webhook "shoot-quota.gardener.cloud" denied the request: requested CPU cores (50) exceed garden quota (40)

If you instead reduce the worker count to 4 (40 cores), the Shoot is created successfully and appears in kubectl get shoots -n garden-prod.

Verification Steps

To confirm the webhook is active and functioning:

• Check that the webhook registration exists: kubectl get validatingwebhookconfiguration shoot-quota -o yaml.
• Watch the webhook pod logs for admission events: kubectl -n gardener logs -l app=gardener-apiserver -c shoot-quota-webhook --follow.
• After applying a Shoot, inspect the AdmissionReview audit log (if audit is enabled) or the event associated with the Shoot: kubectl get events -n garden-prod --field-selector involvedObject.name=over-quota-shoot.
• Compare the observed behavior with the Gardener documentation for your target release (e.g., v1.48) to ensure the quota calculation matches the documented formula.

Limitations and Trade‑offs

While the webhook provides strong guarantees, consider the following:

• Upgrade compatibility: Moving from Gardener v1.47 to v1.48 may change the internal API version used by the webhook. If the Shoot CRD version is upgraded before the webhook, the webhook may fail to deserialize the Shoot spec, causing a temporary denial of all Shoot operations. The recommended upgrade sequence is to first upgrade the Gardener control plane (including the webhook) and then upgrade Shoot CRDs.
• Misconfiguration risk: If the validating webhook configuration is accidentally deleted or the failurePolicy is set to Ignore, quota checks are bypassed, allowing over‑provisioning. Regularly audit the webhook object with kubectl get validatingwebhookconfiguration shoot-quota -o yaml and ensure failurePolicy: Fail.
• Scope: The webhook only enforces quotas defined as ResourceQuota objects in the garden namespace. Custom quota policies (e.g., per‑team limits) must be expressed as such objects; otherwise they are not evaluated.

Actionable Closing

To adopt this guardrail in your landscape:

1. Deploy the default Gardener Helm chart (or operator) in a test namespace and verify the shoot-quota
2. Define garden‑level ResourceQuota objects that reflect your capacity limits (CPU, memory, storage, node count).
3. Apply a test Shoot that intentionally exceeds a quota and confirm the admission denial.
4. Monitor webhook logs and events during normal operations to ensure no unexpected denials occur.
5. Document the upgrade procedure: control plane first, then Shoot CRDs, and validate webhook health after each step.

By relying on Gardener’s admission webhook, you move quota enforcement from periodic checks to an immutable API gate, reducing the chance of accidental over‑provisioning and giving teams predictable resource boundaries.