Designing CPU‑based Auto‑Scaling on Scalingo: Requirements, Minimal Setup and Operational Checks

Problem and Takeaway

You need a Scalingo‑hosted web service to stay responsive under variable traffic without manually adjusting dyno counts. The useful takeaway is a minimal, verified architecture that lets Scalingo’s built‑in autoscaler add or remove dynos based on CPU usage, while keeping the application stateless and observable.

Requirements

• The app must expose a health endpoint (e.g., /healthz) that returns 200 when the service is ready.
• Requests must be idempotent or the app must rely on an external session store so that duplicate processing does not corrupt state.
• The application should be stateless; any state (sessions, caches) lives outside the dyno (Redis, database, object storage).
• You need a Scalingo account with permission to manage formations and add‑ons (owner or collaborator with the Scalingo:admin role).

Minimal Suitable Design

The smallest architecture that satisfies the requirements is a single Scalingo formation running one web process type. The autoscaler add‑on is enabled and configured with CPU thresholds.

Initial Deployment

# Run in your local terminal where the Scalingo CLI is authenticated
scalingo create           # creates the formation
scalingo --app  git push scalingo master   # deploy your code
# Start with a single web dyno
scalingo --app  ps:scale web=1

Enable and Configure Autoscaling

# Enable the autoscaler add‑on (free tier available)
scalingo --app  addons-add autoscaler
# Set scale‑out at 70% average CPU and scale‑in at 30%
scalingo --app  autoscaler:set-thresholds --scale-out 70 --scale-in 30
# (Optional) set a minimum and maximum dyno count
scalingo --app  autoscaler:set-limits --min 1 --max 5

After these commands, Scalingo’s internal autoscaler reads the platform‑provided CPU metric every ~15 s and triggers a scale‑out when the average exceeds the upper threshold for two consecutive periods, and a scale‑in when it stays below the lower threshold for two periods.

Trust and Data Boundaries

The autoscaler operates entirely within Scalingo’s control plane:

• It only reads the application’s CPU metric exposed by the platform; it does not inject code, modify environment variables, or touch your filesystem.
• No data written by your app is accessed or altered by the autoscaler.
• The trust boundary is therefore the metric API; as long as the metric is accurate, the autoscaler’s decisions are confined to adjusting dyno count.

Operational Checks

To confirm that autoscaling behaves as expected, enable visibility and alerting:

Metrics Dashboard

In the Scalingo web UI, open the “Metrics” tab for your app. You should see a chart labeled “CPU %” with horizontal lines at your configured thresholds (70% and 30%).

Event Logging

Scale‑out and scale‑in actions are emitted to the app’s log stream. Retrieve them with:

scalingo --app  logs --lines 100 | grep -i autoscaler

Look for lines like autoscaler: scaling out web from 2 to 3 or autoscaler: scaling in web from 3 to 2.

Alerting (optional)

Create a platform alert on the “autoscaler event” log pattern to notify Slack or email when a scaling action occurs.

Failure Modes and Conditions that Would Change the Design

• Non‑idempotent requests: If a scale‑out creates a duplicate dyno while a request is in flight, and the app is not idempotent, you could see duplicate writes or inconsistent state. Mitigation: enforce idempotency at the business‑logic layer or use an external lock/service.
• Metric mis‑interpretation: Sudden CPU spikes caused by garbage collection or a short‑lived background job may trigger unnecessary scale‑out, raising cost. Mitigation: raise the scale‑out threshold or increase the evaluation period (requires contacting Scalingo support; the CLI does not expose this tunable).
• Scaling delay: The platform typically needs 30‑60 s to provision a new dyno. During a rapid traffic surge, this can cause temporary 502/504 responses. Mitigation: provision a baseline number of dynos that handles expected peak‑minus‑burst load, or use a request‑queuing layer (e.g., Redis‑based rate limiter) to smooth spikes.
• Cost overruns: If the upper threshold is too low or the maximum dyno count is unbounded, the autoscaler could keep adding dynos until the account limit is reached. Mitigation: set a realistic --max limit and monitor monthly usage via the Scalingo billing dashboard.

Verification Steps

Validate the design with a load test that pushes CPU above the upper threshold.

Generate Load

From a machine with network access to your app (e.g., a CI runner or local workstation), run:

# Example using wrk (install via package manager if needed)
wrk -t12 -c400 -d2m https://.scalingo.io/healthz

Adjust threads (-t) and connections (-c) until the Metrics dashboard shows sustained CPU >70%.

Observe Scaling

While the test runs, watch the logs:

scalingo --app  logs --lines 200 | grep -i autoscaler

You should see a scale‑out event within ~45 s of the threshold breach. After stopping the load, CPU should fall below 30% and a scale‑in event follow.

Check Request Success

During the test, periodically request the health endpoint and verify HTTP 200:

curl -s -o /dev/null -w "%{http_code}" https://.scalingo.io/healthz

Any non‑200 response indicates a problem with request handling during scaling.

Post‑Test Cleanup

Stop the load test, wait for scale‑in, then verify the dyno count:

scalingo --app  ps

The output should list the number of web dynos equal to the --min limit you set.

Limitations and Practical Checks

• The autoscaler only reacts to CPU; it does not consider memory, latency, or queue depth. If your bottleneck is I/O‑bound, you may need a custom scaling solution (e.g., using Scalingo’s API to adjust workers based on external metrics).
• Changing thresholds requires CLI commands; there is no rolling‑window configuration exposed in the UI.
• To verify that no orphaned dynos remain after a scale‑in, regularly run scalingo --app ps and ensure the count matches the --min limit.

Rollback (State‑Changing Operation)

Disabling the autoscaler removes the platform‑managed scaling behavior but does not affect your application code or data. To revert to a static dyno count:

# Disable the add‑on (keeps existing dynos)
scalingo --app  addons-remove autoscaler
# Optionally set a fixed number of dynos
scalingo --app  ps:scale web=2   # adjust as needed

After removal, monitor the Metrics dashboard to confirm that CPU no longer triggers automatic scaling events.