Eliminating Cold Starts on Cloud Run: The Min Instances Trade-Off

Cold starts spike p99 latency, but paying for idle capacity solves it.

\n

If your user-facing API experiences unpredictable latency spikes (high p99/p999) due to container startup delays—the 'cold start'—setting a minimum number of instances on Cloud Run is the direct solution. However, this shifts latency risk from startup time to continuous operational cost.

\n

Understanding Min Instances

\n

Min instances forces Cloud Run to keep a specified number of container instances warm and ready to serve traffic, bypassing the latency associated with image pulling and application initialization (which can range from 100 ms to several seconds, depending on your service).

\n

\n
• Mechanism: Configured via --min-instances flag or Service YAML.
\n
• Cost: Each minimum instance incurs continuous charges for allocated vCPU and memory, even when receiving zero traffic.
\n
• Scaling: The service scales from this floor up to a configured max-instances based on request concurrency.
\n

\n

Sizing for Baseline Load, Not Just Average

\n

The correct setting isn't based on average RPS, but on the sustained baseline RPS you must handle without latency penalties. Use this heuristic: Baseline Instances = Ceiling(Sustained Baseline RPS / (Concurrency * RPS per Instance)). Always round up and add a small buffer for burst protection.

\n

Worked Example: E-commerce API

\n

Consider an e-commerce API with a 50 RPS daytime baseline, using a concurrency setting of 80. The required baseline instances are Ceiling(50 / (80 * 25)) = 1, so setting --min-instances=2 provides a small buffer against immediate spikes.

\n

Configuration Example (gcloud):

\n

gcloud run deploy my-api --image=gcr.io/my-project/api --min-instances=2 --concurrency=80 --region=us-central1

\n

Verification: Deploy this, then use curl repeatedly while monitoring Cloud Logging for entries where labels."cloudrun.googleapis.com/cold-start"=true. With min-instances=2, cold starts should only appear after the third concurrent request.

\n

The Cost‑Latency Trade‑Off

\n

The decision is purely economic: Do you prefer predictable, low latency (paying for idle time) or variable, potentially high latency (saving on idle costs)? For SLOs requiring p99 < 200 ms, the cost is usually justified.

\n

Limitation: Billing During Deployments

\n

Crucially, if you use traffic splitting (e.g., 90/10 canary deployment), both the old and new revisions maintain their configured minimum instances simultaneously. This doubles the baseline cost for the duration of the rollout.

\n

Actionable Next Step

\n

Before committing to a fixed min-instances, deploy a test revision with --min-instances=0 and monitor latency percentiles (p99) in Cloud Monitoring. Then, deploy a second test revision with --min-instances=2 and compare the p99 metrics. Only increase the minimum if the latency improvement outweighs the sustained cost increase.