Hibernating Gardener Shoot Clusters: Scaling Dev/Test Kubernetes to Zero While Preserving State

Running a dev or test Kubernetes cluster that sits idle between working hours still incurs cloud VM costs. Deleting and recreating the cluster each time is slow and erases configuration, secrets, and etcd state. Gardener, the Kubernetes multi-cluster management project, offers a middle path: hibernation lets you scale a shoot cluster's worker nodes to zero while preserving its control plane identity and persistent state.

Useful takeaway: you can turn a non-production shoot off at night or on weekends, release the underlying worker VMs, and wake it up in minutes rather than hours, provided you account for version-sensitive API fields and residual cloud charges.

How Gardener hibernation works under the hood

In a typical Gardener deployment, a shoot represents an end-user Kubernetes cluster. Its control plane runs as pods inside a seed cluster, which itself sits on top of a cloud provider infrastructure. The machine-controller-manager within the seed is responsible for creating and managing the worker machines that run your workloads.

When you set spec.hibernation.enabled to true on the Shoot resource, Gardener's reconciliation loops scale the machine-controller-manager deployments down and shut down the worker machine sets. Most control plane components (API server, controllers, etcd) continue running inside the seed, but their worker-node dependencies are removed. The etcd data and cluster configuration persist because they live in the seed's storage, not on the worker VMs.

Wake-up is the inverse: removing the hibernation flag triggers the controller to scale the worker machine sets back up. The API server becomes reachable again, and nodes register -- typically far faster than a full delete-and-create cycle because the cluster identity and secrets are already in place.

Version note: The exact field path and schedule semantics can shift between Gardener releases. Always cross-check spec.hibernation against the API reference for the version deployed in your seed.

Hands-on: enabling hibernation on a shoot

Assume you have a shoot named managed inside a seed called . The following steps demonstrate how to enable hibernation and observe the expected state changes.

Run all commands from a machine with a kubeconfig pointing to the seed cluster and sufficient permissions to modify Shoot resources. Minimum required permissions typically include gardener.shoot.update and read access to machine sets.

kubectl patch shoot  -n  --type merge -p 'spec.hibernation.enabled=true'

After the patch, watch the machine sets drain:

kubectl get machineset -w -n 

You should see the DESIRED and CURRENT replica counts shrink to zero on the worker machine sets associated with the shoot. Simultaneously, the shoot's status.hibernation.state field transitions to Hibernating and later to Hibernated.

In the cloud provider console (AWS EC2, GCP VMware Engine, Azure VMSS, etc.), the underlying VMs belonging to the shoot's machine sets should transition to a stopped or terminated state, releasing the associated IP addresses and storage volumes that are not marked for retention.

Expected checks: kubectl describe shoot shows hibernation.state: Hibernated; kubectl get machineset shows 0/0 replicas for the shoot-related sets; cloud console shows no running worker VMs charged to the shoot's node pool.

Relevant risk: While the shoot is hibernated, any CI pipeline, demo environment, or integration that targets the shoot's API endpoint will fail until wake-up completes. Wake-up latency depends on the cloud provider's VM provisioning time and the speed at which the machine controller can attach storage and join nodes to the cluster.

Trade-offs, limitations, and cost verification

• Residual charges: etcd storage inside the seed, the seed cluster's own control-plane overhead, load-balancer residue, and any cloud resources not managed by the shoot's machine controller will continue to incur charges.
• Schedule interaction: Gardener supports spec.hibernation.schedules with cron expressions and start/end fields. If a schedule is defined alongside a manual enabled: true, the controller evaluates both; unexpected overlaps can cause the cluster to stay up or shut down at times you didn't intend. Test the combination in a non-critical shoot before relying on it for cost governance.
• Version sensitivity: Field names and the depth of the hibernation sub-resource evolve across Gardener releases. A patch that works on one version may be rejected or silently ignored on another.

Practical verification workflow: Before and after a full week of scheduled hibernation, export your cloud provider's cost explorer data for the seed's node pool and shoot-associated resources. Subtract the post-week total from the pre-week baseline; the difference approximates the VM-hour savings attributable to hibernation. Compare this number against your expected savings (worker VM count x hourly rate x hours hibernated) to confirm the feature is delivering the anticipated benefit.

Metric	Before hibernation	After 1 week hibernation	Difference
Monthly VM cost (USD)	$X	$Y	$X-$Y
Running worker machines	N	0 (during hibernation windows)	N machines released

If your organization runs non-production Kubernetes clusters, hibernation can be a low-friction way to reduce infrastructure spend without losing cluster identity or re-provisioning pipelines. Start by enabling enabled: true on a single dev shoot, observe the machine-set drain and control-plane scaling, then measure the cost delta before expanding to a schedule. Remember to account for residual seed charges and verify the feature against the Gardener version running in your environment.