Managing Worker Load with Scalingo Auto-scaling

The Problem: The 'Burst' Bottleneck

Many background worker processes suffer from a binary state: they are either idling and wasting resources or completely overwhelmed by a sudden spike in queue depth, leading to processing delays and potential timeouts. Manually adjusting dyno counts during a traffic surge is reactive and often too slow to prevent a backlog.

The solution is to move from static allocation to a dynamic scaling policy. By automating the scaling of worker dynos based on real-time CPU usage or custom metrics, you can maintain throughput without paying for peak capacity 24/7.

How Scalingo Auto-scaling Operates

Scalingo's auto-scaling service functions as a monitoring loop. It polls a metrics endpoint (typically /metrics) at a defined interval to determine the average resource consumption across your active worker dynos. When the average CPU usage crosses a pre-defined threshold, the system triggers the launch of additional dynos, up to a specified maximum.

A critical part of this lifecycle is the health check. Scalingo does not simply count a dyno as "active" once the process starts. Each new dyno must pass a health check; if it fails, the system terminates that instance and continues the scaling loop until the required capacity of healthy dynos is met.

Configuring Your Scaling Policy

Scaling policies are managed either through the Scalingo CLI (v2.0+) or via a scalingfile in your application root. The policy requires three primary variables: the minimum dyno count (to ensure baseline availability), the maximum dyno count (to cap costs), and the trigger threshold.

Example Configuration

Suppose you have a background worker that processes image uploads. You want at least 2 dynos running, but you want to scale up to 10 if CPU usage exceeds 70%.

Run the following command from your local terminal with the Scalingo CLI installed and authenticated:

scalingo scale worker --min 2 --max 10 --threshold 70

Expected Check: To verify the policy was applied, run scalingo config:get scaling. This should return the current min/max and threshold values associated with your worker process.

Trade-offs: Cost vs. Stability

While auto-scaling solves the capacity problem, it introduces two specific risks: cost spikes and thrashing.

• Cost Spikes: If your threshold is set too low (e.g., 30% CPU), a minor increase in traffic could trigger a maximum scale-out, leading to unexpected billing.
• Thrashing: This occurs when metrics jitter around the threshold. For example, if CPU fluctuates between 69% and 71%, the system may rapidly spin up and terminate dynos. To mitigate this, ensure your metrics are averaged over a meaningful window rather than reacting to instantaneous spikes.

Verifying the Scaling Loop

To confirm your configuration works before a production surge, you can simulate load on a test worker process. Follow these steps:

1. Deploy a worker process with metrics collection enabled.
2. Use a load-testing tool or a script to increase the CPU demand on the worker.
3. Monitor the dyno count via the Scalingo dashboard or by running scalingo metrics in your terminal.
4. Observe if the dyno count increases as the CPU crosses the 70% mark and decreases once the load subsides.

Rollback: If the auto-scaling behavior is unstable or causing cost issues, you can return to a static dyno count by removing the scaling policy via the CLI or deleting the scalingfile and manually setting the scale: scalingo scale worker 2.