Gardener Shoot Control-Plane HA: What `failureTolerance: zone` Changes in the Seed

The control plane lives in the seed, not the shoot

Your shoot's worker nodes might span three zones, but that does not make the Kubernetes control plane highly available. In Gardener, the control plane runs as workloads inside a dedicated namespace on a seed cluster. Unless you explicitly ask for more, kube-apiserver, controller-manager, scheduler, etcd, and related components run as single instances. If the seed node hosting them fails, your shoot is unreachable regardless of how resilient its workers are. The useful takeaway: control-plane HA is a seed-side decision, driven by replica counts and pod topology, and it comes with an etcd quorum trade-off you should understand before you touch production.

The Shoot API exposes control-plane high availability through spec.controlPlane.highAvailability.failureTolerance. When that field is absent or nil, you get the default single-instance control plane. Setting it to zone asks Gardener to spread replicas across zones; node spreads them across nodes. The exact accepted values and defaults are version-sensitive, so confirm the shape against the API reference for your Gardener release before editing production shoots.

etcd sets the real availability ceiling

You can add replicas of kube-apiserver and friends, but etcd is the constraint. Etcd is the distributed key-value store that holds the shoot's cluster state. An HA control plane typically runs an odd number of etcd replicas, commonly three, so the cluster can tolerate the loss of one member.

The trade-off is quorum. Etcd requires a majority of members to agree before it can commit writes. With three replicas, losing two, or losing an entire zone in a three-zone layout, removes quorum. The control plane becomes unavailable until quorum is restored. More replicas increase resilience to isolated failures, but they do not protect against correlated loss that takes out a majority.

Enabling HA also raises seed resource usage and changes upgrade behavior. Rolling updates must preserve quorum, so maintenance windows and pod disruption budgets matter more once replicas exceed one. A partially failed rollout can leave the shoot control plane degraded.

Worked example: moving a development shoot to zone tolerance

Take a non-HA development shoot and request zone failure tolerance. On the Gardener API, edit the Shoot object:

spec:
  controlPlane:
    highAvailability:
      failureTolerance: zone

Run this against the Gardener endpoint or project namespace where the Shoot resource lives. Changing this field triggers control-plane reconciliation and rolling restarts, so do it in a maintenance window and confirm seed capacity first.

After reconciliation, inspect the shoot's control-plane namespace on the seed:

kubectl -n <shoot-control-plane-namespace> get pods -o wide

You need cluster-admin or equivalent RBAC on the seed. Confirm that kube-apiserver and etcd pods now land on distinct nodes and, for zone tolerance, distinct zones. If the seed only has nodes in one zone, the spec cannot deliver zone failure tolerance even when requested.

From a kubeconfig for the shoot, confirm the API server is still healthy:

kubectl cluster-info
kubectl get --raw='/readyz'
kubectl get nodes

Finally, read the Shoot status against the Gardener API with project access to confirm the applied setting and a successful last operation:

kubectl get shoot <name> -o yaml

What HA does not fix

Control-plane HA protects against single-zone or single-node loss in the seed. It does not protect against seed-wide failure, invalid spec changes, or provider control-plane outages. Stronger guarantees still require control-plane migration across seeds and tested etcd backup and restore procedures.

High availability is also not the same as high durability. Without tested etcd snapshots and quorum-aware operations, a majority etcd loss can still cause a shoot outage and risk data loss.

Before you rely on it

• Check the Gardener API reference for your release to confirm the exact controlPlane.highAvailability shape and allowed failureTolerance values.
• Verify the seed actually has nodes in multiple zones with correct provider zone labels.
• Confirm seed capacity for the additional replicas.
• Make the change in a maintenance window and watch the rollout.
• Test etcd backup and restore independently of the HA setting.