Adjusting Cloud Run Concurrency to Match Workload Characteristics

Problem: Default concurrency may not fit your workload

Cloud Run services start with a default concurrency of 80 requests per container. If your code spends most of its time waiting for I/O (calls to APIs, databases, or external services), the default setting can keep CPU under‑utilized while many instances sit idle, leading to higher latency and more instances than necessary. Conversely, for CPU‑bound work, a high concurrency value can cause contention and increase latency.

Thesis: Tuning the concurrency limit aligns resource use with the actual request pattern, reducing latency and instance count while keeping costs predictable

By setting a concurrency value that reflects how much time each request spends waiting versus computing, you let each instance handle multiple requests in parallel. The instance’s CPU is shared among the active requests, so I/O‑bound workloads benefit from higher concurrency, while CPU‑bound workloads stay responsive with lower values.

How Cloud Run allocates CPU with concurrency

Each Cloud Run instance receives a fixed CPU allocation based on the memory you configure. When concurrency is set to 1, the instance uses the full CPU for the single request it processes. When concurrency is N (>1), the CPU time is divided among the N concurrent requests; each request gets a fraction of the CPU cycles. The division is not guaranteed to be perfectly equal, but the overall effect is that higher concurrency reduces per‑request CPU share while increasing the number of requests an instance can handle simultaneously.

Worked example: Tuning a Go HTTP service

1. Deploy a baseline service with the default concurrency (you need gcloud installed and the roles/run.admin role):
   gcloud run deploy hello-concurrency \ --image=gcr.io/google-samples/hello-go:1.0 \ --region=us-central1 \ --allow-unauthenticated \ --project=$PROJECT_ID
2. Generate a steady load using hey (install locally):
   hey -z 1m -c 20 http://$(gcloud run services describe hello-concurrency --platform=managed --region=us-central1 --format='value(status.url)')
3. Observe metrics in Cloud Monitoring: look for run.googleapis.com/request_count, run.googleapis.com/container/cpu/utilization, and run.googleapis.com/instance_count. Note the average latency and the number of instances running at the tested load.
4. Redeploy with a higher concurrency value** (this creates a new revision):
   gcloud run deploy hello-concurrency \ --image=gcr.io/google-samples/hello-go:1.0 \ --region=us-central1 \ --allow-unauthenticated \ --concurrency=50 \ --project=$PROJECT_ID
5. Repeat the load test** with the same hey command and compare the monitoring charts. You should see a reduction in average latency and a lower instance count while CPU utilization per instance rises but stays well below the memory limit.

Trade‑offs and limits

• Memory pressure: each concurrent request holds its own stack and any allocated buffers. Raising concurrency too high can exceed the instance’s memory limit, resulting in OOMKilled errors visible in Cloud Logging.
• Revision traffic split: changing concurrency creates a new revision. If you use traffic splitting, migrate traffic gradually (e.g., in 10 % increments) to avoid sudden latency shifts.
• CPU‑bound workloads: when the service performs heavy computation, sharing CPU among many requests can increase latency. In that case keep concurrency low (typically 4‑8) and rely on autoscaling to add instances as needed.
• Maximum concurrency: Cloud Run allows values up to 1000, but the practical upper bound is often lower due to memory constraints. Always verify memory usage after increasing concurrency.

Actionable checklist

1. Profile your service: measure the proportion of time spent waiting for I/O versus CPU (e.g., with pprof, Cloud Trace, or a simple timing log).
2. Pick a starting concurrency: 80‑100 for pure I/O‑bound code, 4‑8 for CPU‑bound code. Adjust in steps of 10 based on observed metrics.
3. Deploy with the chosen value using the --concurrency= flag and run a representative load test (hey, k6, Locust, etc.) for at least one minute.
4. Check Cloud Monitoring: verify that run.googleapis.com/request_latencies decreases, run.googleapis.com/instance_count drops, and run.googleapis.com/container/cpu/utilization stays below ~80 % to leave headroom for spikes.
5. Watch memory usage: if run.googleapis.com/container/memory/utilization approaches the limit, lower concurrency or increase memory with --memory=.
6. Automate the setting: store the concurrency value in a variable in your CI/CD configuration (e.g., cloudbuild.yaml or GitHub Actions) and apply it on every deployment.
7. Re‑evaluate when the workload changes: new dependencies, added endpoints, or changes in external service latency may shift the I/O/CPU balance, requiring a new concurrency tune.

Tuning concurrency is a low‑effort, high‑impact lever for Cloud Run services. Aligning the limit with your workload’s actual behavior can reduce response times, cut the number of running instances, and make your monthly bill more predictable—without changing a single line of application code.