Tuning Cloud Run Concurrency: Matching --concurrency to Workload Profile and Cost

The default --concurrency 80 on Cloud Run is a compromise that fits few real workloads. It can cause CPU throttling on compute-heavy containers, queue buildup behind single-threaded frameworks, and unnecessary cost on I/O-bound services. The useful rule: set concurrency to match the per-request resource profile of your container and the threading model of your application, then verify with load testing and Cloud Monitoring.

What --concurrency actually controls

In Cloud Run, --concurrency sets the maximum number of requests a single container instance will accept at the same time. It is not a service-wide limit. If you set --concurrency 50 and Cloud Run runs three instances, your service can handle up to 150 simultaneous requests.

When a request arrives and all existing instances are at their concurrency limit, Cloud Run queues the request and starts a new instance if capacity allows. The default queue timeout is about 10 seconds; after that the client receives a 503. This queue is why a misconfigured concurrency value turns into user-visible latency.

A worked deployment and why the numbers fit

Consider a JSON API that does moderate CPU work per request—parsing payloads, validating data, writing to a database. A single vCPU cannot process dozens of these requests in parallel without queuing inside the container. The following deployment reserves enough CPU for the expected parallelism and keeps one warm instance to avoid cold starts on the first request:

gcloud run deploy myservice \
  --image gcr.io/project/myservice \
  --region us-central1 \
  --concurrency 50 \
  --cpu 2 \
  --memory 2Gi \
  --min-instances 1 \
  --max-instances 100

Run this from Cloud Shell or any terminal with the Cloud SDK installed and the run.admin IAM role (or equivalent) on the target project. Replace gcr.io/project/myservice and us-central1 with your image and region.

The choice of 50 concurrent requests on 2 vCPU assumes each request uses roughly 1/25th of a CPU on average. If your container is CPU-intensive, drop concurrency or raise --cpu. If it is I/O-bound—waiting on network calls or database queries—you can often run 50–100+ concurrent requests per vCPU.

How autoscaling reacts to your setting

Cloud Run adds instances when average concurrency utilization across active instances exceeds roughly 60% of the configured limit. With --concurrency 50, scaling begins around 30 simultaneous requests per instance. The 60% threshold is a buffer: it gives new instances time to start before the queue fills.

If traffic spikes faster than instances can start, requests wait in the queue. Because the timeout is fixed at about 10 seconds, a spike that outpaces autoscaling produces 503s rather than slow successes. --max-instances caps total scale-out and therefore total cost; --min-instances keeps warm capacity but bills for idle time at the same vCPU and memory rates.

CPU, memory, and the threading trap

Concurrency is a ceiling, not a guarantee of parallel execution. A single-threaded synchronous framework—Flask without Gunicorn workers, for example—can accept 80 connections but process them one at a time. The default --concurrency 80 then creates an internal queue inside the container, not throughput. Latency rises linearly with load, and the instance looks healthy while requests time out.

For CPU-bound work, the rule of thumb is at least 1 vCPU per 10–20 concurrent requests. Below that ratio, the Linux Completely Fair Scheduler throttles the container, latency climbs, and the runtime may kill the instance if memory also spikes. Memory is a hard limit: exceeding --memory triggers an OOM kill and restart, losing in-flight requests.

Long-lived connections: gRPC and WebSockets

Streaming protocols change the math. Each gRPC stream or WebSocket connection counts as one concurrent request for its entire lifetime. A service holding 100 idle WebSocket connections at --concurrency 80 has no capacity left for new HTTP requests.

For these workloads, lower concurrency to the number of simultaneous connections you expect—often 10–20—and use the Gen2 execution environment. Gen2 is the default for new services and provides better HTTP/2 and connection handling. If your service uses VPC connectors, confirm egress settings match Gen2 behavior before migrating.

Verifying the setting and watching the metrics

After deploying, confirm the applied value:

gcloud run services describe myservice \
  --region us-central1 \
  --format='value(spec.template.spec.containerConcurrency)'

Then load-test with a tool such as hey to observe behavior under pressure:

hey -c 20 -n 200 https://myservice-xxxxx-uc.a.run.app

Replace the URL with your service endpoint. Watch three Cloud Monitoring metrics: run.googleapis.com/container/request_count for arrival rate, run.googleapis.com/container/cpu.utilization for throttling, and run.googleapis.com/container/billable_instance_time for cost correlation. In logs, look for Container reached concurrency limit or Request queued; both indicate saturation.

Limits and rollback

Concurrency cannot exceed 1000, and the default is 80. Requests that wait longer than roughly 10 seconds receive a 503. Custom domains and Identity-Aware Proxy add network latency, which counts against that queue budget.

Because gcloud run deploy changes state, roll back by redeploying the previous revision or updating the concurrency value:

gcloud run services update myservice \
  --region us-central1 \
  --concurrency 80

If you need to revert to an earlier revision entirely, use gcloud run revisions list to find the revision name, then gcloud run services update-traffic myservice --region us-central1 --to-revisions REVISION_NAME=100.