Balancing Cold Starts and Cost: Choosing Cloud Run CPU Allocation

The Latency vs. Cost Dilemma

When deploying a containerized application to Cloud Run, the most immediate performance hurdle is the "cold start"—the delay experienced when a request triggers the creation of a new container instance. While increasing min-instances can mitigate this, it introduces a constant cost. The real engineering decision lies in the CPU Allocation setting: whether to allocate CPU only during request processing or to keep it always allocated.

Choosing the wrong mode often leads to either unexpected billing spikes or erratic application performance, particularly for services that rely on background threads or heavy initialization logic.

Request-Based vs. Always-Allocated CPU

By default, Cloud Run uses CPU allocated during request processing. In this mode, the CPU is throttled to nearly zero when the instance is not actively handling an HTTP request. This is highly cost-effective for sporadic traffic but can cause issues if your app performs asynchronous work (like sending a confirmation email after the HTTP response is sent) or maintains a persistent connection pool.

CPU always allocated keeps the CPU active regardless of whether a request is being processed. This is essential for workloads that require background processing, predictable warm-up times, or high-performance connection pooling to a database. Because the CPU is always active, the instance doesn't "sleep" as deeply, reducing the latency of the first request after a period of inactivity.

Decision Matrix: Which Mode to Use?

Requirement	CPU during requests	CPU always allocated
Cost Optimization	High (Pay only for active requests)	Lower (Pay for instance uptime)
Background Tasks	Not supported (CPU throttled)	Supported
DB Connection Pooling	Connections may drop/timeout	Stable connection maintenance
Cold Start Impact	Significant for new instances	Reduced (when paired with min-instances)

Worked Example: Configuring for Background Processing

Imagine a service that receives a file upload, returns a 202 Accepted response immediately, and then processes the file in a background thread. With default CPU settings, the background thread would be throttled the moment the response is sent, causing the processing to hang or fail.

To solve this, you must change the CPU allocation. Run the following command using the gcloud CLI (ensure you have the cloudrun.services.update permission):

gcloud run services update [SERVICE_NAME] \
  --no-cpu-throttling \
  --region [REGION]

Expected Result: The --no-cpu-throttling flag enables "CPU always allocated." Now, the container continues to execute logic after the HTTP response is delivered to the client.

Verification: To check if the setting applied, run gcloud run services describe [SERVICE_NAME] and look for the cpu-throttling property. It should be set to false.

The Trade-off: Cost and Scaling

The primary limitation of "CPU always allocated" is the billing model. You are charged for the entire lifecycle of the instance, not just the milliseconds it spends processing a request. If you combine this with a high min-instances count to eliminate cold starts, you are effectively paying for a server that is always on, moving away from the "serverless" cost advantage.

Additionally, if your application is highly concurrent (handling many requests per instance), the cost difference is negligible. However, for low-traffic services, this setting can increase costs by orders of magnitude.

Actionable Summary

Before deploying your next revision, audit your application's lifecycle. If your app needs to maintain a heartbeat to a database or perform cleanup tasks after a response is sent, use --no-cpu-throttling. If your app is a pure "request-in, response-out" function with low traffic, stick to the default request-based allocation to keep costs at a minimum.