Discloud Autoscaling for Node.js: Picking CPU Thresholds That Avoid the 502 Window

The problem: one Node.js process, one saturation point

Node.js executes JavaScript on a single thread using an event loop, the mechanism that processes incoming requests one at a time. When a request triggers CPU-intensive work—image manipulation, large JSON transforms, cryptographic signing—the event loop blocks and every other request waits. On Discloud, horizontal autoscaling is meant to relieve that pressure by adding more container replicas rather than enlarging a single instance, but it does not react instantly. If your threshold and replica limits are misconfigured, the platform either scales too late and returns 502 errors, or scales too eagerly and burns budget on idle containers.

The useful takeaway: treat Discloud's autoscaling policy as a capacity contract, not a set-and-forget switch. You need to know the reaction window, the per-service boundaries, and the hard ceiling before you point production traffic at it.

How Discloud decides to add a replica

Discloud's automatic horizontal scaling watches aggregate CPU and memory utilization for each service. When CPU usage exceeds the configured threshold for a sustained period—platform documentation describes a default of 70% for two consecutive minutes—Discloud launches an additional container replica and distributes incoming traffic through its internal load balancer. Scaling decisions are made independently per service, which means a web tier and a background job tier in the same project can run different thresholds and replica limits.

Two behaviors matter operationally. First, the user-defined maximum replica count is a hard limit; once you reach it, further load produces queueing and increased latency rather than new instances. Second, scaling events are written to the service's log stream, so you can inspect them with the Discloud CLI.

Worked example: an Express service with a 50% CPU trigger

Suppose you have a Node.js Express API deployed on Discloud that occasionally spikes when a report endpoint runs. You want Discloud to react earlier than the default 70% threshold, but you do not want more than three replicas. The plan is to configure a 50% CPU threshold with a maximum of three replicas, generate sustained load, and watch the replica count change.

Run the following from your terminal where the Discloud CLI is installed. You need deploy and log permissions on the target project.

# Check current replica count for your service
discloud ps

# Stream scaling events and application logs
discloud logs --follow

In your Discloud project configuration, set the autoscaling policy for this service to a maximum of three replicas and a CPU threshold of 50%. Exact field names depend on your manifest version; verify the current keys in the platform documentation before deploying.

From any machine with network access to the deployed endpoint, generate sustained traffic with a load-testing tool such as autocannon:

# Replace with your actual Discloud service URL
autocannon -c 100 -d 180 https://<your-service>.discloud.app

Expected checks:

• discloud ps should show one replica at baseline.
• After CPU stays above 50% for the sustained window, a second replica should appear.
• discloud logs --follow should show scaling messages; the exact wording varies by platform version, but look for entries indicating a replica was added.
• When the load stops, the replica count should return to baseline after a cooldown period.

Risk: if traffic arrives faster than the reaction window, the original replica may saturate before the new one passes health checks, producing temporary 502 responses.

The over-provisioning trap: aggregate metrics and bursty jobs

Discloud bases scaling on aggregate CPU and memory. That model fits steady request traffic, but it struggles with uneven workloads. Imagine a background worker that sits at 5% CPU for 55 minutes and then runs a heavy batch import at 90% for five minutes. If the threshold is set aggressively low, the service scales up during the batch and stays scaled until the cooldown expires, even though the web tier never needed the extra capacity. Because scaling is per-service, the standard mitigation is to split web workers and background jobs into separate Discloud services, each with its own threshold and replica ceiling.

Limitations and a safer rollout

Autoscaling is not a substitute for capacity planning. The maximum replica count is absolute: in the example above, a fourth concurrent user wave will queue behind three instances. The reaction window also means sudden launches can outrun the scaler. If your traffic pattern includes predictable spikes—marketing pushes, cron-driven report generation—pre-warm the service by raising the baseline replica count before the event, or lower the CPU threshold so the sustained window starts earlier.

A practical verification loop: deploy with a conservative threshold, run a moderate load test for at least five minutes, and watch discloud ps. If you see 502 errors before a new replica appears, the threshold is too high or the baseline instance is too small. If replicas appear while the service is idle, the threshold is too low or the workload is bursty enough to warrant a separate service tier. To roll back the change, disable autoscaling in the service configuration and redeploy; the service returns to a fixed replica count.