Cloud Run Scaling Decision: Automatic Concurrency vs Fixed Instances for Cost-Optimized Workloads

Decision framing: cost ceiling vs latency for bursty container workloads

The practical problem is choosing a scaling mode that keeps a bursty, containerized service under a $0.05 per GB-hour cost ceiling while respecting Cloud Run limits: a 60-second request timeout and a maximum container size of 2 GB. The useful takeaway is that automatic scaling with a high per-instance concurrency can match bursty demand without paying for idle capacity, provided you accept cold-start latency and guard against 429 rejections.

Min instances, max instances, and concurrency are the three levers that determine cost and latency. Min instances keeps containers warm. Max instances caps scale-out. Concurrency controls how many requests a single instance handles in parallel.

Constraints that shape the choice

Cloud Run enforces a 60-second request timeout for standard Cloud Run. Long-running work must move to Cloud Run for Anthos or be broken into Cloud Tasks.

Container size affects per-GB-hour billing. A 2 GB container that runs for one hour costs more than a 512 MB container for the same duration. Cost is pay-per-request plus per-GB-hour active instance time.

Burst traffic that exceeds configured concurrency is rejected with HTTP 429. Clients need exponential back-off.

Scaling options compared

Trade-offs in practice

Automatic scaling offers cost efficiency and rapid scale-out for bursty traffic. With min instances set to 0, you pay only when requests arrive. Raising concurrency to 80 reduces the number of instances needed for a given request rate, which helps stay under a per-GB-hour cost ceiling.

The cost is cold-start latency when traffic resumes after idle. The first request after a scale-up pays container start time.

Fixed instances guarantee warm containers and predictable latency. You set minInstances to N and pay for those instances continuously, even during low traffic. This is wasteful for spiky workloads.

Cloud Run for Anthos adds network and IAM considerations. VPC connector and secret access policies can affect egress cost and startup time. Verify service configuration with gcloud describe.

Concrete configuration for automatic scaling with high concurrency

Deploy with an explicit max instances and concurrency target. Run the command from a machine with gcloud authenticated and the Run Admin role, e.g., roles/run.admin, on the target project.

gcloud run services create my-svc --image gcr.io/PROJECT/my-img --region us-central1 --max-instances 80 --concurrency 80

Placeholders: PROJECT is your GCP project ID, my-img is the container image, us-central1 is the region. The flags set the scaling cap and per-instance request parallelism.

Risks: setting concurrency too high can increase per-request memory pressure inside a 2 GB container. Max instances 80 caps cost but can still be hit by sustained load, leading to 429 responses. If traffic is sustained, consider a lower concurrency with more instances for steadier latency.

Validation checklist

Confirm the scaling settings are applied.

gcloud run services describe my-svc --region us-central1 --format json

Check the scaling field for minInstances, maxInstances, and concurrency. Required permission: roles/run.viewer.

Observe instance behavior. In Cloud Logging, filter for run.googleapis.com resource type and look for instance start events. On Anthos, pod startup timestamps in the run-system namespace can be inspected.

Measure request latency. Send a single probe with timing.

curl -w '\n%{time_total}' https://my-svc.run.app

Compare observed latency against your cold-start threshold. For load shape testing, a tool such as hey can generate concurrent requests, e.g., 100 requests with 20 concurrent clients, and you can observe latency distribution and instance count in Cloud Console.

Limitations: Cloud Run standard has the 60-second timeout. Automatic scaling does not eliminate cold starts on idle. Cost per GB-hour depends on actual instance active time and memory allocation. Verify billing in Cloud Billing with the resource filter for Cloud Run.