Cloud Run scale-to-zero exhibits latency spikes after idle periods

Answer the question first

For a workload with an average request rate of N requests per hour, the cost‑effective minInstances is either 0 or 1. Choose minInstances = 1 only when the expected latency penalty of a cold start (based on the observed distribution) exceeds the latency threshold you can tolerate; otherwise keep minInstances = 0 to minimise cost.

Confirmed facts

• Cloud Run bills only for the time an instance is alive (instance‑seconds). Setting minInstances = k reserves k instances continuously, adding a baseline cost of k × price_per_instance_second × 3600 per hour.
• When minInstances = 0, each request that arrives after an idle period incurs a cold start. The latency of that cold start equals the time to pull the container image, initialize the runtime, and load the application code.
• Changing minInstances creates a new revision; traffic is split during rollout, which can temporarily affect latency.

Likely explanation & steps for this case

1. Measure the cold‑start latency distribution:
   • Enable request‑level logging in Cloud Run.
   • Export the container_start_time metric (or compute latency as request_timestamp - container_start_timestamp).
   • Collect a representative sample (e.g., several hundred cold starts) and derive key percentiles (median, 90th, 95th).
2. Define your latency threshold (e.g., maximum acceptable 95th‑percentile latency). This is the missing diagnostic detail; without it we cannot decide whether the observed cold‑start latency is acceptable.
3. Compare the threshold to the measured distribution:
   • If the chosen percentile (e.g., 95th) of cold‑start latency is ≤ your threshold, keeping an instance warm offers no latency benefit; set minInstances = 0.
   • If the percentile exceeds the threshold, the expected latency cost per request justifies warming an instance; set minInstances = 1.
4. Validate the decision over time:
   • Track run.googleapis.com/request_count to verify the average request rate stays near N.
   • Monitor run.googleapis.com/container/start_latency (or the derived cold‑start latency) to ensure spikes remain below the threshold.
   • Watch run.googleapis.com/instance_count to confirm the configured minInstances is respected.

Cloud Monitoring metrics to validate latency‑cost balance

• run.googleapis.com/request_count – requests per hour (to confirm N).
• run.googleapis.com/container/start_latency – latency of each container start (cold‑start proxy).
• run.googleapis.com/instance_count – number of live instances (ensures minInstances is honored).
• run.googleapis.com/request_latencies – overall request latency distribution (to verify end‑user latency stays within target).