Diagnosing and Fixing High Latency from Cold Starts and Concurrency Limits in Google Cloud Run

Recognizable Symptom

You notice request latency spikes above 2 seconds during low or bursty traffic, see frequent "container starting" messages in logs, and observe that the active instance count stays at zero or one while requests are queued.

Cause & Diagnostic Table

Observed Condition	Likely Cause
Latency > 2 s, low request rate, instance count = 0	Cold start due to insufficient warm instances
CPU utilization at or near limit during request processing	CPU allocation too low for workload
Concurrency setting lower than peak request rate	Requests queued waiting for a free concurrency slot
Frequent container restarts, OOM‑like logs	Memory limit too low
Request timeout errors after ~60 s	Request processing exceeds default timeout

Ordered Checks

1. Verify traffic pattern: In Cloud Monitoring, view the "Requests per second" metric for the service over the last 15 minutes. Note if traffic is bursty or consistently low.
2. Check instance count: Look at the "Instance count" metric. If it stays at 0 or 1 while requests arrive, warm instances are lacking.
3. Inspect CPU usage: Examine the "CPU utilization" metric. Values consistently ≥ 80 % indicate the allocated CPU is a bottleneck.
4. Review concurrency setting: Retrieve the current concurrency with gcloud run services describe SERVICE --region REGION --format='value(spec.containerConcurrency)'. Compare to observed peak request rate.
5. Check memory limit: Use the same describe command to see spec.template.spec.containers[0].resources.limits.memory. Look for OOMKilled events in logs.
6. Confirm timeout: Verify the service timeout (spec.timeoutSeconds) and note any 504 errors in Cloud Logging.

Fixes Tied to Findings

• If instance count is low despite traffic: Increase the minimum number of instances to keep containers warm.

  gcloud run services update SERVICE \
    --region=REGION \
    --min-instances=2 \
    --quiet
  

  Required permission: roles/run.admin or roles/editor on the project. Risk: Constantly running instances incur cost even when idle. Check: After change, watch "Instance count" rise to the new minimum and latency drop.
• If CPU utilization is high: Raise CPU allocation (note: CPU is allocated only during request processing unless "CPU always allocated" is enabled).

  gcloud run services update SERVICE \
    --region=REGION \
    --cpu=2 \
    --quiet
  

  Risk: Higher CPU increases per‑instance cost; monitor usage.
• If concurrency is below peak rate: Increase concurrency up to the maximum allowed (80 for CPU‑allocated instances).

  gcloud run services update SERVICE \
    --region=REGION \
    --concurrency=80 \
    --quiet
  

  Risk: Over‑provisioning can cause overload if the container cannot handle parallel requests; test with a load generator first.
• If memory limit causes restarts: Increase memory and redeploy.

  gcloud run services update SERVICE \
    --region=REGION \
    --memory=512Mi \
    --quiet
  

  Note: Changing memory requires a new container image revision; the command triggers a redeploy.
• If processing exceeds timeout: Raise the timeout (max 60 s for HTTP, up to 600 s for internal traffic with Cloud Run for Anthos; for fully managed Cloud Run the limit is 60 s). If you need longer, consider moving work to background tasks or Cloud Tasks.

Escalation Criteria

• Latency remains > 2 s after applying all feasible configuration changes.
• Cost increase from raising min‑instances or CPU exceeds budget thresholds.
• Logs show repeated crash loops or OOMKilled despite memory increase.
• Concurrency at the maximum (80) still results in queuing, indicating the container cannot handle parallel load; consider refactoring the code or moving to a different compute option (e.g., Cloud Run for Anthos with higher concurrency limits or Compute Engine).

Verification

1. After each change, open Cloud Monitoring and view the "Request latency" metric (99th percentile) for at least 5 minutes.
2. Confirm that "container starting" log entries drop significantly.
3. Run a quick load test (e.g., hey -z 2m -q 10 -c 20 https://SERVICE-REGION.a.run.app) and verify that latency stays under your SLO.
4. Check the cost estimate in the Cloud Run console under the "Edit and deploy new revision" dialog to ensure it aligns with expectations.

Limitations

• Concurrency cannot exceed 80 for CPU‑allocated instances in fully managed Cloud Run.
• Minimum instances incur continuous cost; there is no free tier for warmed instances.
• CPU always allocated can increase latency for very short requests because the CPU is kept running idle.
• Memory changes require a new image revision; you cannot adjust memory without redeploying.

Example Configuration

Suppose a service image‑processor in region us-central1 shows:

• Peak request rate ≈ 70 rps
• Current concurrency = 10
• CPU = 1
• Min‑instances = 0
• Memory = 256Mi

Applying the following changes addresses the observed cold‑start latency and queuing:

gcloud run services update image-processor \
  --region=us-central1 \
  --min-instances=2 \
  --cpu=2 \
  --concurrency=80 \
  --memory=512Mi \
  --quiet

After deployment, monitor latency; you should see the 99th‑percentile drop from > 2 s to under 500 ms under the same load.

Rollback Steps (if needed)

To revert a specific change, re‑run the update command with the previous value. For example, to restore concurrency to 10:

gcloud run services update image-processor \
  --region=us-central1 \
  --concurrency=10 \
  --quiet

Repeat for each parameter you wish to roll back. Verify metrics after each rollback to confirm the effect.