Optimizing Cloud Run Concurrency to Prevent OOMKilled Crashes

The Balance Between Concurrency and Resource Exhaustion

The primary challenge when scaling Cloud Run is balancing concurrency—the number of simultaneous requests a single container instance handles—against the allocated CPU and memory. If concurrency is set too high, a spike in simultaneous requests can exhaust the container's memory, leading to an OOMKilled (Out of Memory) error, which crashes the instance and drops all active connections.

The takeaway: Do not rely on the default concurrency settings for memory-intensive applications. You must align your concurrency limit with the actual memory footprint of a single request to ensure stability during traffic bursts.

How Concurrency Works in Cloud Run

Unlike traditional serverless functions that handle one request per instance, Cloud Run allows a single instance to process multiple requests in parallel. This reduces cold starts (the delay when a new instance boots up) and lowers costs by maximizing resource utilization.

When the number of active requests reaches the configured concurrency limit, Cloud Run stops sending traffic to that instance and triggers the autoscaler to spin up a new revision. If the autoscaler cannot keep up, the platform may return 429 Too Many Requests responses.

Example: Configuring Concurrency and Resources

Consider a Python application that processes PDF documents. Each request requires approximately 150MB of RAM. If the container is allocated 1GB of memory, setting concurrency to 100 would likely crash the instance (100 requests × 150MB = 15GB required).

To stabilize this, you should set the concurrency limit to a value that leaves headroom for the application runtime and overhead. In this case, a limit of 5 is a safer starting point.

Run the following command using the gcloud CLI to deploy with specific concurrency and memory limits:

# Run this from your local terminal with project-owner or cloud-run-admin permissions
gcloud run deploy pdf-processor \
  --image gcr.io/project-id/pdf-app \
  --concurrency 5 \
  --memory 1Gi \
  --cpu 1 \
  --region us-central1

Expected Result: The service will now limit each instance to 5 simultaneous requests. Once the 6th concurrent request arrives, Cloud Run will attempt to route it to a different instance or start a new one.

Common Engineering Mistakes

• Over-provisioning Concurrency: Setting a high concurrency limit to save money on instances, only to have the service crash under load. This creates a "death spiral" where crashing instances put more load on remaining instances, causing them to crash as well.
• Ignoring Memory Overhead: Forgetting that the container OS and application runtime (e.g., JVM or Node.js) consume memory before a single request is even processed.
• Assuming Linear Scaling: Assuming that doubling CPU automatically allows for doubling concurrency. Memory is often the actual bottleneck in stateless workloads.

Verification and Diagnostics

To verify if your concurrency settings are correct, you can simulate a load spike and monitor the platform's reaction.

1. Load Testing

Use a tool like hey to send concurrent traffic to your service URL:

# Send 100 total requests, 10 at a time
hey -n 100 -c 10 https://pdf-processor-xyz.a.run.app

2. Monitoring for Failures

Check the Google Cloud Console Logs Explorer for the following indicator:

• Search for "Memory limit exceeded" or check the instance metrics for oom_killed events.
• If you see OOMKilled, either decrease the concurrency limit or increase the allocated memory.

3. Validating Scaling Behavior

Observe the Instances graph in the Cloud Run console. If the instance count increases linearly as you increase the -c (concurrency) flag in your load test, the concurrency limit is being respected.

Limitations

Cloud Run has platform-wide maximums for concurrency and timeouts. While you can increase the request timeout (default 60 seconds), doing so increases the duration a request occupies a "concurrency slot," effectively reducing the throughput of a single instance. Always balance timeout settings with concurrency limits to avoid artificial bottlenecks.