PodDisruptionBudgets: The Kubernetes Guardrail That Can Block Your Node Drains

You kick off a routine node drain during a cluster upgrade, and kubectl drain simply hangs. No fatal error, no progress—just retry after retry. The culprit is often a PodDisruptionBudget (PDB), a resource that caps how many pods can be voluntarily evicted at once. The useful takeaway is that PDBs are guardrails for the eviction API, not the scheduler, and their percentage math can leave a node undrainable if you configure them carelessly.

What PDBs Actually Gate

A PodDisruptionBudget limits voluntary disruptions: node drains, cluster upgrades, and administrative pod deletions that go through the eviction API. When you run kubectl drain, the tool asks the API server to create an Eviction object for each pod. The PDB controller checks whether evicting that pod would drop the number of healthy pods below the budget. If it would, the eviction API rejects the request with HTTP 429 Too Many Requests, and kubectl drain blocks or retries until the budget is satisfied or a timeout is reached.

PDBs do not stop the scheduler from placing replacement pods on other nodes, nor do they prevent a Deployment from creating new replicas. They also do nothing for involuntary disruptions such as node hardware failures, kernel panics, or OOM kills. Think of a PDB as a circuit breaker for planned maintenance, not a high-availability guarantee.

The Percentage Rounding Trap

Both minAvailable and maxUnavailable accept absolute integers or percentages, but percentages are always rounded down to the nearest whole pod. If you set maxUnavailable: 50% on a three-replica Deployment, the controller calculates 1.5 and rounds down to 1. Only one pod can be evicted at a time, not 1.5. The same rounding applies to minAvailable: a 50% budget on three replicas means at least one pod must stay available, so two disruptions are permitted. The asymmetry matters when you are trying to drain a node that hosts multiple replicas.

A Worked Example: Three Replicas, One Budget, One Drain

Imagine a stateless web application running three replicas. You want to ensure at least two pods remain healthy during maintenance, so you create a PDB with minAvailable: 2. The manifests look like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  namespace: default
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
      - name: nginx
        image: nginx:1.25
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web-pdb
  namespace: default
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: web

Apply both from a machine with kubectl configured against the target cluster. You need permission to create Deployments and PodDisruptionBudgets in the namespace.

kubectl apply -f deployment.yaml
kubectl apply -f pdb.yaml

Once the pods are running, identify a node that hosts one of them:

kubectl get pods -l app=web -o wide

Pick a node name from the NODE column and attempt a drain. Run this from an administrative context with permission to create evictions and delete pods:

kubectl drain <node-name> --ignore-daemonsets --delete-emptydir-data

Because the PDB requires two healthy pods and the drain would leave only two on other nodes, the eviction API should reject the request with HTTP 429. The command will block or retry rather than delete the pod immediately. To confirm the budget state, run:

kubectl get pdb web-pdb -o wide

Inspect the ALLOWED DISRUPTIONS, CURRENT HEALTHY, and DESIRED HEALTHY columns. If ALLOWED DISRUPTIONS is 0, the drain cannot proceed until another pod becomes healthy elsewhere or you relax the budget.

If you need to stop the drain and restore the node to service, interrupt the command and uncordon the node. This is the rollback step because kubectl drain changes the node's schedulable state:

kubectl uncordon <node-name>

Risk: --delete-emptydir-data discards data stored in emptyDir volumes on the drained node. The --disable-eviction flag bypasses PDB enforcement entirely and should be used only when you accept the availability risk. The --grace-period flag controls how long pods have to terminate gracefully, but it does not change whether the eviction API accepts the request.

When the Guardrail Becomes the Blocker

The most common failure mode is setting minAvailable equal to the replica count. If you have three replicas and minAvailable: 3, every pod is un-evictable. Node upgrades, autoscaling, and routine maintenance stall indefinitely. Use maxUnavailable instead when you want to allow rolling replacements; it expresses the same intent with less risk of deadlock.

Label selectors are another silent trap. A typo in the PDB selector leaves your workload unprotected, while an orphaned PDB with no matching pods consumes no budget but also provides no safety. Audit regularly:

kubectl get pdb -A

Finally, PDB controller behavior and eviction API semantics can vary across Kubernetes minor versions and managed control planes. Always validate against your specific cluster release. And remember: a PDB cannot keep a single-replica service online. It reduces the blast radius of voluntary disruptions, but it is not a substitute for multiple replicas or anti-affinity rules.

Actionable Takeaways

• Use maxUnavailable rather than minAvailable when you want to permit rolling updates and node drains.
• Never set minAvailable to the full replica count unless you intend to block all voluntary evictions.
• Check ALLOWED DISRUPTIONS with kubectl get pdb before starting a drain window.
• Verify PDB selector labels match your pod templates exactly.
• Confirm eviction API and drain flag behavior against the Kubernetes documentation for your cluster version.