Traefik Mesh Weighted Traffic Splitting for Docker Swarm: Canary Releases with Service Labels

Useful Answer

Traefik Mesh supports weighted traffic splitting for Docker Swarm services through service labels, letting you route a percentage of east‑west HTTP requests to a newer service version while the rest continues to the stable version. This enables canary releases without external load balancers or Kubernetes‑style CRDs. The mesh controller reads Docker service labels on manager nodes and configures the sidecar proxies to split traffic according to the declared weights.

How the Mechanism Works

Traefik Mesh runs a controller on each Swarm manager. When you deploy or update a service with specific labels, the controller receives Docker events, reconciles the desired state, and pushes routing rules to the mesh proxies running on every node. For weighted splitting, you deploy two (or more) Swarm services that share the same logical mesh service name but carry different traefik.mesh.service.weights labels. The proxy then distributes incoming HTTP requests proportionally.

Worked Configuration Example

Assume you have a service called myapp currently running version v1. You want to send 10 % of traffic to a new v2 image while keeping 90 % on v1. Both services must be attached to the same overlay network that Traefik Mesh uses (typically traefik-mesh).

# Deploy stable version (v1)
docker service create \
  --name myapp-v1 \
  --network traefik-mesh \
  --label traefik.enable=true \
  --label traefik.mesh.service.name=myapp \
  --label traefik.mesh.service.weights=90 \
  --label traefik.http.services.myapp.loadbalancer.server.port=8080 \
  myorg/myapp:v1

# Deploy canary version (v2)
docker service create \
  --name myapp-v2 \
  --network traefik-mesh \
  --label traefik.enable=true \
  --label traefik.mesh.service.name=myapp \
  --label traefik.mesh.service.weights=10 \
  --label traefik.http.services.myapp.loadbalancer.server.port=8080 \
  myorg/myapp:v2

Key label breakdown:

• traefik.mesh.service.name=myapp — logical mesh service name shared by both versions.
• traefik.mesh.service.weights — integer weight; the controller normalizes weights across all services sharing the same mesh name.
• traefik.http.services.myapp.loadbalancer.server.port — tells the mesh proxy which container port to forward to (must be identical on both versions).

After both services are running, any request to http://myapp (resolved via Mesh DNS) will be split 90/10 between the two task sets.

Verification Steps

Run a quick distribution test from a container attached to the mesh network:

docker run --rm --network traefik-mesh curlimages/curl:latest \
  sh -c 'for i in {1..100}; do curl -s http://myapp/version; done | sort | uniq -c'

Expect output close to 90 responses showing v1 and 10 showing v2. Check the Traefik dashboard (if enabled) or proxy logs on a manager node to confirm backend registration and weight values:

docker service logs traefik_mesh_controller 2>&1 | grep -i weight

Limits and Constraints

Protocol Scope

Weighted splitting applies only to HTTP traffic handled by the mesh router. TCP, UDP, or raw TLS passthrough services do not honor traefik.mesh.service.weights labels; they fall back to round‑robin across healthy tasks.

Version Sensitivity

Label names and weight semantics changed between Traefik Mesh 1.x and 2.x. The example above targets Traefik Mesh ≥ 2.3 (Traefik Proxy ≥ 2.6). Older releases used traefik.backend.weight or required a separate traefik.http.services.myapp.weighted construct. Always verify the label reference for your exact version.

Health Checks Are Separate

Assigning weight to an unhealthy backend does not stop traffic. You must also define health‑check labels (e.g., traefik.http.services.myapp.loadbalancer.healthcheck.path=/health) so the proxy can remove failing tasks from the pool.

Controller Reload Impact

Each label change triggers a controller reconciliation and proxy reload. In large Swarms (hundreds of services), frequent weight adjustments can cause brief latency spikes. Batch changes when possible.

Common Mistakes

Rollback Procedure

If the canary misbehaves, set its weight to 0 and the stable version to 100:

docker service update \
  --label-rm traefik.mesh.service.weights \
  --label-add traefik.mesh.service.weights=0 \
  myapp-v2

docker service update \
  --label-rm traefik.mesh.service.weights \
  --label-add traefik.mesh.service.weights=100 \
  myapp-v1

The controller picks up the label changes within seconds; no container restarts required.

Practical Checklist Before Production

1. Confirm Swarm mode is active (docker info | grep -i swarm).
2. Verify Traefik Mesh controller runs on managers (docker service ps traefik_mesh_controller).
3. Ensure both service versions share the same overlay network and publish the same container port.
4. Define health‑check labels on each version.
5. Test weight distribution with a scripted curl loop before shifting production traffic.
6. Monitor proxy logs and dashboard for backend registration and weight application.

Weighted splitting in Traefik Mesh is a Swarm‑native, label‑driven mechanism that works well for HTTP canary releases when you respect its protocol limits, version‑specific label syntax, and the need for independent health checks.