Envoy's Token Bucket Rate Limit Filter: Configuration, Keying, and Common Pitfalls

If you need per-client or per-path request throttling inside Envoy without standing up a separate rate-limit service, the local token bucket filter (envoy.filters.http.local_ratelimit) is the right tool. It enforces limits in-process, adds no network hop, and can be updated at runtime through xDS. The trade-off is that each Envoy instance keeps its own buckets, so limits are per-instance, not global.

How the filter decides

The filter maintains a bucket per key. Each request consumes one token; tokens refill continuously at the configured fill_rate up to max_tokens. When a request arrives and the bucket is empty, the filter rejects it with HTTP 429 before the request is dispatched upstream. Because enforcement happens in the filter chain, your upstream only ever sees allowed traffic — which is exactly what you want for capacity protection.

The key is derived from request attributes. If you omit keying, every request shares one global bucket. If you key on a descriptor such as a header value or source IP, each distinct value gets its own bucket.

A worked configuration

The following listener-level HCM fragment limits each client (identified by the x-client-id header) to 50 requests per second with a burst allowance of 100, while capping overall traffic at 1000 rps:

http_filters:
- name: envoy.filters.http.local_ratelimit
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
    stat_prefix: http_local_rate_limiter
    token_bucket:
      max_tokens: 1000
      tokens_per_fill: 1000
      fill_interval: 1s
    filter_enabled:
      runtime_key: local_rate_limit_enabled
      default_value:
        numerator: 100
        denominator: HUNDRED
    filter_enforced:
      runtime_key: local_rate_limit_enforced
      default_value:
        numerator: 100
        denominator: HUNDRED
    descriptors:
    - entries:
      - key: client
        value: per-client
      token_bucket:
        max_tokens: 100
        tokens_per_fill: 50
        fill_interval: 1s
    rate_limits:
    - actions:
      - request_headers:
          header_name: x-client-id
          descriptor_key: client
- name: envoy.filters.http.router
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

Two things to notice. First, the top-level token_bucket is the global fallback; the descriptors entry overrides it when the request matches the client descriptor produced by the rate_limits action. Second, filter_enabled/filter_enforced let you run the filter in shadow mode (counting but not rejecting) by setting filter_enforced to 0 — a safe way to validate keying before turning enforcement on.

Where this runs and what you need

Apply this in your Envoy bootstrap or via LDS (Listener Discovery Service) from your control plane. No special host permissions are needed beyond normal Envoy config reload rights. If you push config via xDS, updates propagate within the normal sync interval — no restart required, which makes this suitable for runtime capacity adjustments during an incident.

Verifying enforcement

After applying the config, send a burst that exceeds the bucket, for example with a loop of 150 rapid requests carrying the same x-client-id. Expect the first ~100 to pass and the remainder to return 429. Then confirm on the Envoy admin interface (default port 9901):

curl -s localhost:9901/stats | grep http_local_rate_limiter

Look for counters such as http_local_rate_limiter.http_local_rate_limit.enabled and ...enforced (or rate_limited depending on version). Access logs will also show 429 responses with response flag RL (rate limited) when envoy.filters.http.local_ratelimit rejects a request — grep for RL plus the path to confirm the right traffic is being caught.

Limits and common mistakes

• Per-instance, not global. With 10 Envoy replicas and a 50 rps per-client bucket, a client effectively gets up to 500 rps across the fleet. If you need a fleet-wide limit, use the global rate limit service (envoy.filters.http.ratelimit with an RLS backend) instead.
• Key collisions. Reusing one key for unrelated traffic patterns forces them to contend on a shared bucket. Keep keys scoped to the dimension you actually want to limit (per client, per path, per tenant), not a mix.
• Unbounded key cardinality. Keying on something with unbounded values (a user-ID header from the internet, or a key accidentally built from variable query strings) creates a bucket per value, growing memory and producing inconsistent limiting. Only key on attributes you control and can bound.
• Missing bucket config. If a descriptor matches but has no token_bucket and the top-level bucket is absent or unlimited, requests pass unchecked. Always set an explicit global bucket as a safety net.
• Filter ordering. The local rate limit filter must run before the router, and after any filter that mutates the attributes your key depends on. If a later filter rewrites the header you keyed on, ordering mistakes can bypass the intended limit. Keep it early in the chain, after authentication if you key on identity.

Capacity planning notes

Set max_tokens to the burst you can tolerate and tokens_per_fill to the sustainable rate. A large bucket with a slow fill absorbs deploy-time stampedes without raising steady-state throughput. Start in shadow mode (filter_enforced: 0), watch the enabled counter against real traffic for a day, then enforce. That gives you evidence-based limits instead of guesses, and the xDS path means tightening them later is a config push, not a rollout.