Choosing a Rate Limiting Strategy in Traefik Proxy for Multi-Replica HTTP Services

The Problem: Replica-Local Limits Under Round-Robin Traffic

When you run multiple Traefik Proxy replicas behind a load balancer without session affinity, the built-in rateLimit middleware enforces its counter independently on each instance. A client that sends 100 requests per minute may be allowed 100 requests per replica, not 100 requests globally. If you run three replicas, the effective ceiling becomes 300 requests per minute under round-robin distribution. The decision is whether to accept that approximation or introduce shared state to enforce a true global limit.

Useful takeaway: Choose the built-in in-memory middleware when you can tolerate per-replica limits and brief relaxation during rolling updates. Choose a shared-state backend only when you need a hard, consistent global ceiling per client and can accept the operational cost of an external dependency.

Decision Constraints

• Stateless replicas with no session affinity, so any replica can receive any client request.
• Predictable 429 Too Many Requests responses at the edge before backend ingress.
• Minimal added latency in the request path.
• Limits must apply consistently per client IP or per API key across all replicas.
• Operational simplicity is valued; extra infrastructure must be justified.

Option Comparison

Aspect	Built-in rateLimit middleware	Shared-state limiter (Redis or external service)
State location	In-memory counters per Traefik instance	Central store reachable by all replicas
Global consistency	Per replica; effective limit multiplies by replica count	True global limit per client key
Latency overhead	Negligible; local memory access	Network round trip to the shared store
Operational complexity	None beyond Traefik itself	Additional service to deploy, monitor, and secure
Restart behavior	Counters reset; limits relax temporarily after pod restart	State persists across Traefik restarts
Failure mode	Fails closed (limits enforced locally)	Depends on store availability; may fail open or closed

Trade-offs

The built-in middleware is the fastest and simplest option. It has no external dependency and adds virtually no latency. However, under round-robin traffic its per-replica nature means a client can exceed the intended global limit by a factor equal to the number of replicas. During a rolling update, the new replica starts with empty counters, temporarily allowing a burst of additional requests.

A shared-state limiter solves the fairness problem. Every replica consults the same store, so burst accounting and average rates are accurate globally. The cost is a network round trip on every request and a new failure domain: if Redis or the external service is unavailable, you must decide whether to fail open (allow traffic) or fail closed (reject traffic).

Per-IP limiting is straightforward with Traefik's client IP detection. Per-API-key limiting requires extracting a header value as the rate limit key, which introduces a middleware ordering dependency: authentication must run before rate limiting. Otherwise, unauthenticated requests can consume quota for a given API key, or requests with forged headers can poison a victim's bucket.

Concrete Implementation: Per-Client Limits at the Edge

The following examples use Traefik's dynamic configuration. Adjust the provider syntax (Kubernetes CRD, Docker label, or file) to match your deployment. Field names below assume Traefik Proxy v2.5 or newer; verify exact availability against your deployed version.

Per-IP limiting behind a trusted proxy

When Traefik sits behind a load balancer or CDN, the direct socket peer is the proxy, not the client. Set ipStrategy.depth to the number of trusted proxy hops so Traefik reads the correct client IP from the X-Forwarded-For header.

http:
  middlewares:
    edge-ratelimit:
      rateLimit:
        average: 100
        burst: 150
        sourceCriterion:
          ipStrategy:
            depth: 1
  routers:
    api-router:
      rule: "Host(`api.example.com`)"
      service: api-service
      middlewares:
        - edge-ratelimit

In this configuration, average is the sustained rate and burst is the additional requests allowed in a short spike. Traefik's built-in middleware evaluates average in requests per second.

Per-API-key limiting with authentication first

To limit by API key, use sourceCriterion.requestHeaderName and place the authentication middleware before the rate limiter in the router's middleware list.

http:
  middlewares:
    api-auth:
      basicAuth:
        users:
          - "placeholder:$apr1$H6uskkkW$IgXLP6ewTrSuBkTrqE8Hj/"  # replace with your own hash
    api-key-ratelimit:
      rateLimit:
        average: 100
        burst: 150
        sourceCriterion:
          requestHeaderName: X-API-Key
  routers:
    api-router:
      rule: "Host(`api.example.com`)"
      service: api-service
      middlewares:
        - api-auth
        - api-key-ratelimit

Middleware order is sequential. If api-key-ratelimit ran before api-auth, requests bearing an X-API-Key header would consume that key's quota before the credential is validated.

Validation and Diagnostics

Confirm that the middleware is attached and that throttling triggers at the expected rate.

Inspect the middleware chain

Open the Traefik dashboard (typically exposed on the api@internal service) and navigate to the router. Verify that edge-ratelimit or api-key-ratelimit appears in the middleware list in the correct order.

Check access logs for 429 responses

Enable access logs in the static configuration. Then run the following from a machine with cluster access:

kubectl logs -n <traefik-namespace> deploy/<traefik-deployment> --tail=200 | grep " 429 "

Where to run: any workstation with kubectl configured.
Permissions: read access to Traefik pod logs.
Placeholders: replace <traefik-namespace> and <traefik-deployment> with your values.
Expected check: after exceeding average + burst, you should see HTTP 429 status codes in the log lines.
Risk: high request volume can produce large log output; pipe through head or redirect to a file if needed.

Load test from a single source

Send sustained requests from one client IP or with one API key across all replicas. Compare the number of accepted requests against the configured average. With the built-in middleware and multiple replicas, the accepted count will approximate average × replica count under round-robin distribution, confirming the per-replica behavior.

Limitations and Operational Risks

• In-memory counters reset on restart. During a rolling update, the replacement replica begins with an empty bucket, temporarily allowing additional requests.
• Client IP detection depends on proxy configuration. If trustedIPs and forwarded header settings are incorrect, Traefik may rate-limit the proxy IP instead of the client, causing all users to share one bucket.
• Middleware ordering is security-sensitive. Rate limiting by API key before authentication lets unauthenticated traffic consume or exhaust a key's quota.
• Shared-state limiter adds a dependency. Redis or an external service becomes part of the critical request path; plan for its availability and decide on fail-open versus fail-closed behavior.
• Built-in average is per second. If you need minute- or hour-based windows, the built-in middleware alone is insufficient without an external limiter.

Practical Result Check

After applying the configuration, send a short burst of requests from a single client and confirm that responses transition from 200 to 429 once the burst is exhausted. Then inspect the Traefik dashboard to ensure the middleware is attached to the intended router and that no other middleware is overriding the response. If 429s appear too early or too late, verify the depth value and the header name used for the source criterion.