When to Use a Pod Disruption Budget with minAvailable for Critical Workloads

Problem: How to guarantee minimal availability during node maintenance

When a cluster node is drained for upgrades or failures, Kubernetes evicts the pods on that node. For critical services you want to guarantee that a certain number of replicas stay running, but you also need to allow the cluster to perform routine maintenance automatically. The question is: should you add a Pod Disruption Budget (PDB) with a minAvailable constraint, or rely on manual coordination?

Decision & Constraints

Decision: Adopt a PDB with minAvailable for any workload that requires a guaranteed number of replicas during voluntary or involuntary disruptions.

Constraints:

• Cluster version >= 1.2 (PDB introduced in 1.2).
• Deployment selector must match the pods you want to protect.
• Do not use --force drains unless you have an additional safety mechanism.
• Be aware that a strict minAvailable can delay node drains, impacting upgrade speed.

Option Comparison

Option	Key Feature	Pros	Cons
No PDB	Manual coordination during node drains.	Simple, no extra API objects.	High risk of accidental downtime; requires human intervention.
PDB – minAvailable	Declarative guarantee of a minimum number of replicas.	Automated enforcement, works with voluntary drains and autoscaler.	Can block necessary node upgrades if the desired count cannot be met.
PDB – maxUnavailable	Allows up to a specified number of evictions.	Flexible, easier for workloads that can tolerate a few lost pods.	Requires careful calculation to avoid dropping below desired availability.

Trade‑offs Explained

Adding a PDB increases the number of objects in the cluster and adds a small amount of overhead to the API server. It also forces the scheduler to consider the budget during eviction decisions, which can delay node drains. However, the declarative nature means you never have to remember to coordinate manually; the cluster will enforce the rule automatically.

Choosing minAvailable vs maxUnavailable is a matter of how much availability you can tolerate. minAvailable is straightforward: you specify the number of replicas that must stay alive. maxUnavailable is more flexible but requires you to calculate the complement (total replicas minus maxUnavailable) to understand the actual availability guarantee.

In high‑availability scenarios, minAvailable is often the safer choice because it provides a clear numeric target. For workloads that can tolerate a temporary drop in replicas, maxUnavailable may be preferable to avoid blocking node upgrades.

Concrete Implementation

1. Define the PDB

Create a YAML file named pdb.yaml that targets the deployment selector. Replace app: web with your own labels.

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web-pdb
spec:
  selector:
    matchLabels:
      app: web
  minAvailable: 2

What it does: Guarantees that at least two pods matching app: web stay running during any voluntary or involuntary disruption.

2. Apply the PDB

Run the following command on a machine with kubectl configured for the target cluster. You need cluster‑role‑admin or equivalent permissions.

kubectl apply -f pdb.yaml

Check that the resource was created:

kubectl get pdb web-pdb

3. Validate the Constraint

Simulate a node drain to confirm the PDB is enforced. Replace <node-name> with the actual node you want to drain.

kubectl drain <node-name> --ignore-daemonsets --delete-local-data

During the drain, kubectl get pods -l app=web should show that no more than the allowed number of pods are terminated. If you had 5 replicas, you should see at least 2 running after the drain completes.

4. Inspect PDB Status

After the drain, describe the PDB to see events and status:

kubectl describe pdb web-pdb

Look for lines like Current: 3 and Desired: 2 confirming the minimum is respected.

Potential Pitfalls & Mitigation

• Older Clusters: If your cluster is older than 1.2, PDB is unavailable. Upgrade or use manual coordination.
• Cluster Autoscaler Interaction: Autoscaler may delay node drains because the PDB blocks eviction. Monitor drain times and adjust minAvailable if necessary.
• Forceful Drains: Using --force bypasses PDB. Protect critical workloads with additional safeguards such as pod disruption budgets on the node or external monitoring.

Practical Checks After Deployment

1. Run kubectl get pdb regularly to verify the minAvailable value matches your expectation.
2. During scheduled maintenance windows, perform a dry‑run drain on a non‑critical node and confirm the PDB behaves as intended.
3. Set up a simple alert in your monitoring system: if the number of running pods falls below minAvailable, trigger a notification.

Summary

For critical workloads that must maintain a specific number of replicas during node maintenance, a Pod Disruption Budget with a minAvailable constraint offers a declarative, automated safeguard. It trades a small amount of flexibility in upgrade speed for a clear availability guarantee, simplifying operations and reducing the risk of accidental downtime.