Envoy Global Rate Limiting: Stopping Cascading Failures at the Edge

Your backend can handle 500 requests per second. Then a client bug, a retry storm, or a sudden traffic spike pushes it to 2,000 — and instead of rejecting excess work, the service slows down, queues back up, and takes its dependencies down with it. Local rate limiting inside each app instance doesn't fix this, because each instance only sees its own slice of traffic. This is the problem Envoy's global rate limiting is built for: one shared quota, enforced consistently across every proxy in your fleet.

Why local limits aren't enough

Envoy supports local (per-instance) rate limiting, which is simple and fast but counts tokens independently on each proxy. If you run ten Envoy replicas and each allows 100 rps, your effective limit is 1,000 rps — and it silently scales every time you add a replica. Global rate limiting moves the counting to a central service, so "100 requests per minute for this API key" means exactly that, no matter how many proxies front the traffic.

How the pieces fit together

The architecture has three parts:

• Envoy (data plane): for each matched request, it builds one or more descriptors — key/value pairs like (remote_address, 203.0.113.7) or (header_match, premium) — and asks the rate limit service whether the request is within quota.
• Rate Limit Service (RLS): a separate gRPC service implementing Envoy's rate limit API (the reference implementation is envoyproxy/ratelimit). It keeps counters, typically backed by Redis, using a token-bucket-style algorithm, and answers OK or OVER_LIMIT.
• Configuration: route-level rate_limits actions define which descriptors get generated; the RLS has its own config defining the actual quota for each descriptor pattern.

This split is deliberate: Envoy stays dumb and fast, while quota policy lives in one place you can update without rolling your proxies.

A worked example

The following limits requests per source IP. First, the route action that generates a descriptor from the client address (this goes in your route configuration):

route_config:
  name: local_route
  virtual_hosts:
  - name: backend
    domains: ["*"]
    routes:
    - match: { prefix: "/api" }
      route: { cluster: backend_service }
      rate_limits:
      - actions:
        - remote_address: {}

Then the HTTP connection manager needs the rate limit filter before the router filter:

http_filters:
- name: envoy.filters.http.ratelimit
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
    domain: my-api
    rate_limit_service:
      grpc_service:
        envoy_grpc:
          cluster_name: rate_limit_cluster
      transport_api_version: V3
- name: envoy.filters.http.router

The domain namespaces your quotas so one RLS can serve many applications. On the RLS side (for the reference implementation, a YAML file loaded via CONFIG_PATH), the matching quota looks like:

domain: my-api
descriptors:
  - key: remote_address
    rate_limit:
      unit: minute
      requests_per_unit: 100

With this in place, any single source IP exceeding 100 requests per minute gets an HTTP 429 from Envoy, and the request never reaches your backend.

Verifying it actually works

Don't assume the wiring is correct — test it. From a machine that can reach the proxy, send a burst of requests and watch the status codes:

for i in $(seq 1 150); do
  curl -s -o /dev/null -w "%{http_code}\n" http://<envoy-host>:<port>/api/test
done | sort | uniq -c

You should see a transition from 200 (or whatever your backend returns) to 429 around the 100-request mark. Also check Envoy's admin interface (commonly http://<envoy-host>:9901/stats) for counters like cluster.rate_limit_cluster.upstream_rq_2xx — if those are zero, Envoy isn't even reaching the RLS, and your limits are decorative. The RLS logs will show the descriptors it receives, which is the fastest way to debug a descriptor that doesn't match your quota config.

The trade-off: a network hop and a failure mode decision

Global rate limiting adds a synchronous gRPC call to every matched request. With a healthy, co-located RLS this is typically sub-millisecond to a few milliseconds, but it's real latency and a real dependency. You must decide what happens when the RLS is down or slow. Envoy's filter defaults to fail-open (requests pass when the RLS errors), which favors availability; setting failure_mode_deny: true flips to fail-closed, which favors protection but turns an RLS outage into a site outage.

Practical mitigations: run at least two RLS replicas behind a load-balanced Envoy cluster, size Redis for the counter load, and alert on RLS error rates the same way you alert on backend errors. Also note the counter is only as precise as your descriptor design — a remote_address limit behind a NAT or CDN will group many users into one bucket, so prefer authenticated identity headers when available.

Where to start

Pick one expensive endpoint, add a route-level descriptor for it, and run the RLS in shadow-observation mode first: keep fail-open behavior and watch the RLS logs and Envoy's ratelimit.over_limit stats to see what would be rejected. Once the descriptor matches your expectations and the numbers look sane, tighten the quota and treat the 429s as a feature your backend will thank you for.