Local vs. Global Rate Limiting in Envoy: A Practical Engineering Decision

The Retry-Storm Problem

A single misbehaving client can saturate your API. Maybe it is a retry loop after a timeout, or a scraper ignoring backoff. If you only limit requests inside your application, the damage is already done: the connection was accepted, a worker thread was occupied, and downstream dependencies were called. You need a gatekeeper at the edge.

Thesis: Pick the Right Filter for the Job

Envoy gives you two different rate-limiting filters, and they are not interchangeable. envoy.filters.http.local_ratelimit enforces a token bucket inside each Envoy process. envoy.filters.http.rate_limit asks an external gRPC service for a decision. The first protects one proxy; the second enforces a cluster-wide quota. Choosing wrong either wastes a network hop or lets clients bypass your limits by load-balancing across replicas.

Local Limits: Fast, Simple, Per-Process

The local filter is a pure in-memory token bucket. There is no external dependency and no added latency beyond a memory lookup. It is the right tool when your goal is to keep one Envoy instance from being overwhelmed, or when you run a single proxy.

Here is a route-level configuration that allows 10 requests per second with a burst of 5. Place this under an HTTP filter in your listener, before the router.

typed_config:
  "@type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
  stat_prefix: http_local_rate_limiter
  token_bucket:
    max_tokens: 5
    fill_interval: 0.1s
    tokens_per_fill: 1
  filter_enabled:
    runtime_key: local_rate_limit_enabled
    default_value:
      numerator: 100
      denominator: HUNDRED
  filter_enforced:
    runtime_key: local_rate_limit_enforced
    default_value:
      numerator: 100
      denominator: HUNDRED

If you run three Envoy replicas behind a load balancer, the effective ceiling is roughly 30 requests per second, because each process tracks its own bucket. That is fine for overload protection, but it is not a global user quota.

Global Limits: Consistency at the Cost of a Hop

When you need a hard limit across the entire fleet—say, 100 requests per minute per API key—you need the global filter. Envoy does not talk to Redis directly. Instead, it sends a RateLimitRequest over gRPC to a rate limit service. That service, such as the Lyft ratelimit project or a custom implementation, can use Redis or another store to keep shared counters.

The Envoy side is simple: define the filter, point it at a gRPC cluster, and attach descriptors to your route so the service knows which counter to check.

http_filters:
- name: envoy.filters.http.rate_limit
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.rate_limit.v3.RateLimit
    domain: public_api
    rate_limit_service:
      grpc_service:
        envoy_grpc:
          cluster_name: rate_limit_cluster
      timeout: 0.25s
- name: envoy.filters.http.router
  typed_config: {}

On the route, add descriptors that tell the service which attributes matter. This example limits by the client's remote address.

route:
  cluster: service_cluster
  rate_limits:
  - actions:
    - remote_address: {}

The rate limit service receives the descriptor, checks its store, and returns OK or OVER_LIMIT. If the service is unreachable, Envoy's behavior depends on failure_mode_deny. Set it to true to fail closed (reject traffic when the service is down) or leave it false to fail open.

Worked Example: Per-IP Quota with Burst Protection

Suppose you want to allow 20 requests per second per client IP on /search, but tolerate bursts up to 40. You would use the global filter with a descriptor on remote_address, and configure the rate limit service to return a quota with a burst multiplier. The Envoy route config stays the same; the policy lives in the service.

To verify the setup, send traffic from a single source and watch the stats endpoint. Envoy exposes counters on the admin port (default 9901).

# Run from the Envoy host or a sidecar with admin access
curl http://localhost:9901/stats/prometheus | grep rate_limit

Look for rate_limit_exceeded_total incrementing as you cross the threshold. If the counter stays at zero while you receive 429s from the application instead of Envoy, the filter is not matching the route or the descriptor is empty.

Trade-offs and Limitations

• Latency: The global filter adds a network round trip to every request. In a tight latency budget, measure the p99 impact before enabling it broadly.
• Operational surface: A distributed limit means another service to deploy, monitor, and scale. Redis becomes a dependency for correctness, not just caching.
• Descriptor complexity: Composite descriptors (header + path + IP) are powerful but easy to misconfigure. An empty descriptor can collapse all clients into one bucket.
• Failure modes: Local limits fail safe by design—the bucket is in memory. Global limits can fail open or closed; choose deliberately.

Actionable Checklist

1. Decide whether you need per-proxy overload protection (local) or a global user quota (global).
2. For local limits, start with a conservative tokens_per_fill and raise it based on observed 429 rates.
3. For global limits, deploy a gRPC rate limit service with a Redis backend before enabling the filter in production.
4. Set failure_mode_deny explicitly. The default may not match your risk tolerance.
5. Alert on rate_limit_exceeded_total and on rate limit service health, not just Envoy health.

Rate limiting is an edge decision with downstream consequences. Match the filter to the guarantee you actually need, and measure before you enforce.