Preventing Application Downtime with Kubernetes Horizontal Pod Autoscaler (HPA)

The Scaling Gap: When Static Replicas Fail

Deploying a fixed number of pods is a gamble. If you over-provision, you waste cloud spend on idle resources. If you under-provision, a sudden spike in traffic leads to CPU throttling, increased latency, and eventually, 5xx errors as your application crashes under the load. The goal is to maintain a consistent performance profile regardless of the request volume without manual intervention.

The Horizontal Pod Autoscaler (HPA) solves this by dynamically adjusting the number of pod replicas in a Deployment or ReplicaSet. It transforms your scaling strategy from a manual guessing game into a reactive control loop based on real-time telemetry.

How the HPA Control Loop Operates

The HPA does not "watch" your pods in real-time; instead, it operates on a periodic polling cycle. It queries the Metrics Server—a cluster-wide aggregator of resource usage data—to determine the current utilization of your pods.

The decision to scale is driven by a specific ratio formula:
desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )]

For example, if you have 2 pods running at 100% CPU but your target is 50%, the HPA calculates 2 * (100/50) = 4. It will then signal the Deployment controller to spin up two additional pods to distribute the load.

Prerequisites for Successful Scaling

HPA will fail to function if two critical components are missing:

• Metrics Server: This is the data source. If kubectl top pods returns an error, your HPA cannot see the resource usage and will remain in an state.
• Resource Requests: HPA calculates percentages based on the requests field in your pod specification, not the limits. If a pod has no CPU request defined, the HPA has no baseline to calculate a percentage, and scaling will not trigger.

Implementation Example: CPU-Based Scaling

In this scenario, we target a deployment where we want to maintain an average CPU utilization of 50% across all pods, with a minimum of 2 pods for high availability and a maximum of 10 to prevent runaway costs.

1. Define the Deployment with Resource Requests

Run this on your local machine with cluster-admin permissions. Note the resources.requests section; this is the anchor for the HPA.

# deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        resources:
          requests:
            cpu: \"100m\" # 0.1 CPU core
          limits:
            cpu: \"500m\"

2. Apply the HPA Configuration

Apply the following manifest to create the autoscaling object:

# hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50

3. Verification and Testing

To verify the HPA is active, run:
kubectl get hpa web-app-hpa

To test the scaling, use a load-generation tool (like Apache Benchmark or Hey) to send a flood of requests to the service. Observe the TARGETS column in the HPA output. You should see the current percentage rise above 50%, followed by the REPLICAS count increasing.

The "Flapping" Problem and Stabilization

A common risk with HPA is flapping—a cycle where the HPA adds a pod, the average CPU drops immediately, and the HPA deletes the pod, only for the CPU to spike again. This creates instability and puts pressure on the container runtime.

Kubernetes manages this via a stabilization window. By default, the HPA remembers the highest recommended replica count over a period (usually 5 minutes for scale-down). It will not terminate pods until the load has remained consistently low for the duration of that window, ensuring that brief dips in traffic don't trigger premature downsizing.

Critical Limitations

HPA is not a complete scaling solution on its own. If your HPA requests 10 pods but your physical nodes are full, those pods will remain in a Pending state. To solve this, you must pair HPA with a Cluster Autoscaler, which monitors for pending pods and provisions new virtual machines from your cloud provider.

Additionally, CPU-based scaling is a proxy for load. For I/O-bound applications (like a database proxy) or applications with memory leaks, CPU utilization may remain low while the app is failing. In these cases, you must implement Custom Metrics (using tools like Prometheus and the Custom Metrics API) to scale based on request queue length or memory usage.

Closing Action

Review your current deployments. If you have pods without resources.requests, you are currently blind to your actual utilization and unable to use HPA. Start by adding requests to your most critical services, then implement a conservative HPA with a wide stabilization window to ensure stability during traffic swings.