Using Traefik Mesh TrafficSplit for Weighted Canary Deployments

The problem: rolling out a new version without risking the whole service

When you need to push a new version of a microservice, a straight‑forward rollout can expose all users to bugs. You want a way to send only a fraction of traffic to the new build, watch its behavior, and then shift more traffic as confidence grows.

Thesis: Traefik Mesh implements the SMI TrafficSplit spec, which lets you weight traffic between multiple backends without changing application code

Traefik Mesh injects a sidecar proxy into each pod. The sidecar reads a TrafficSplit custom resource defined by the Service Mesh Interface (SMI) spec (split.smi-spec.io/v1alpha2). The resource lists backends and a weight for each; the proxy distributes incoming HTTP/HTTPS requests according to those weights, while leaving non‑HTTP traffic untouched.

Note: Traefik Mesh was announced as deprecated by Traefik Labs around 2023 and is no longer actively maintained. The feature described here reflects the behavior of the last released version; operators should consider current alternatives for new projects.

Worked example: moving from v1 to v2 of a web API

1. Deploy the two versions as separate Kubernetes Services, each selecting its own pod set via labels (e.g., api-v1 selects version=v1, api-v2 selects version=v2).
2. Create a TrafficSplit that initially sends 90 % to v1 and 10 % to v2:

apiVersion: split.smi-spec.io/v1alpha2
dkind: TrafficSplit
metadata:
  name: api-split
  namespace: production
spec:
  service: api          # the front‑end service that clients call
  backends:
    - serviceName: api-v1
      weight: 90
    - serviceName: api-v2
      weight: 10

Apply the manifest with kubectl apply -f traffic-split.yaml. This requires permission to create CRDs in the cluster and that sidecar injection is enabled for the pods backing api-v1 and api-v2.

Gradual rollout: after verifying v2’s health (e.g., via observability or a simple curl loop), update the same resource:

    - serviceName: api-v1
      weight: 50
    - serviceName: api-v2
      weight: 50

Later shift to 0/100 to promote v2 fully.

Trade‑offs and limitations

• Protocol scope: weighted splitting only applies to HTTP/HTTPS traffic. For pure TCP routes the SMI TrafficSplit is ignored, and connections are load‑balanced across all endpoints of the referenced service.
• Operational overhead: you must monitor per‑backend metrics (request count, latency, error rates) to decide when to adjust weights. If sidecar injection is missing on any pod, the split is bypassed entirely for that pod.
• Session affinity: the TrafficSplit resource does not provide sticky sessions; affinity must be configured at the sidecar or ingress layer if required by your application.

Verification and next steps

To confirm the split is working:

1. Generate a steady stream of requests, e.g., for i in {1..100}; do curl -s http://api.example.com/health; done.
2. Differentiate responses by a version header, hostname, or any other attribute you set in each backend.
3. Check the sidecar’s admin endpoint (typically exposed on localhost:9090) for metrics; look for counters that correspond to each backend (exact metric names depend on the Traefik Mesh version, consult its documentation).
4. Optionally enable access logs on the sidecar and verify that log entries contain the correct destination service name according to the configured ratio.

If the distribution deviates significantly, verify that sidecar injection is present on all pods (kubectl get pods -l app=api -o jsonpath='{.items[*].spec.containers[*].name}' should list the proxy container) and that the TrafficSplit resides in the same namespace as the service.

Actionable closing: start with a small canary weight (5‑10 %), observe key SLOs, and incrementally increase the weight only when the new version meets your criteria. This limits blast radius while providing a fast rollback path—simply set the weight back to 0 for the canary.