Managing Canary Releases with Cloud Run Traffic Splitting

The Risk of the 'Big Bang' Deployment

Deploying a new version of a service by replacing 100% of the traffic instantly is a high-risk maneuver. If a critical bug exists in the new build, every single user experiences the failure simultaneously, leading to immediate downtime and urgent rollbacks.

The solution is a canary release: routing a small percentage of live traffic to a new version (the canary) while the majority remains on the stable version. In Google Cloud Run, this is achieved through Traffic Splitting, which decouples the deployment of code from the exposure of that code to users.

How Cloud Run Handles Revision Routing

Cloud Run treats every deployment as an immutable Revision. A revision is a snapshot of the container image, environment variables, and resource limits. When you update a service, you aren't modifying an existing instance; you are creating a new revision.

The routing layer sits in front of these revisions. By adjusting the traffic split, you tell the Google Front End (GFE) what percentage of incoming requests should be directed to which revision ID. Because revisions scale independently, a canary receiving only 5% of traffic will scale its instances accordingly, maintaining the cost-efficiency of scale-to-zero without overloading the new version.

Implementing a Gradual Rollout

To execute a canary deployment, you must first deploy the new version without assigning it 100% of the traffic. This allows you to verify the health of the revision in the production environment before exposing it to the general public.

Example: Shifting 10% of traffic to a new revision

Run the following command from your local terminal or Cloud Shell. You will need the cloud-platform and run.services.update permissions.

# 1. Deploy the new version but assign it 0% traffic initially
gcloud run deploy my-service --image gcr.io/my-project/my-app:v2 --no-traffic

Once the deployment is complete, note the Revision name (e.g., my-service-00002-abc). Now, split the traffic between the stable version (my-service-00001-xyz) and the canary:

# 2. Split traffic: 90% to stable, 10% to canary
gcloud run services update-traffic my-service --to-revisions my-service-00001-xyz=90,my-service-00002-abc=10

Verification: To confirm the routing is active, run gcloud run services describe my-service. Look for the traffic section in the output to ensure the percentages match your configuration.

Monitoring and Decision Points

A canary release is only useful if you have the telemetry to decide whether to proceed. Because each revision is a distinct resource, you can filter logs and metrics specifically for the canary.

• Log Filtering: In Cloud Logging, use the filter resource.type="cloud_run_revision" AND resource.labels.revision_name="my-service-00002-abc" to isolate errors occurring only in the new version.
• Latency Checks: Compare the request latency of the canary against the stable baseline using Cloud Monitoring.

Trade-offs and Operational Limits

Traffic splitting is powerful, but it introduces specific behaviors that engineers must account for:

• Request Persistence: Traffic splitting happens at the request level, not the session level. A single user may hit the stable version on one request and the canary on the next. If your update involves breaking changes to a database schema or session state, you must ensure backward compatibility.
• In-Flight Requests: When you shift traffic (e.g., from 10% to 100%), requests that were already being processed by the old revision are allowed to complete. There is no instant “kill” of active connections.
• Minimum Allocation: To keep a revision active and receiving traffic, it must be assigned at least 1%. To stop all traffic to a version, it must be explicitly set to 0%.

Actionable Rollback Strategy

The primary advantage of this architecture is the speed of recovery. If the canary shows a spike in 5xx errors, you can revert to the stable version instantly without redeploying code.

# Immediate rollback to 100% stable
gcloud run services update-traffic my-service --to-revisions my-service-00001-xyz=100

This operation only updates the routing table in the GFE, making the recovery nearly instantaneous regardless of the size of your container image.