Envoy Rate Limiting: Local Filter vs External Rate Limit Service

You need to stop a client from overwhelming an upstream, but you are not sure whether to count requests inside Envoy or call an external rate-limit service. The practical takeaway is simple: use the local in-process filter when the limit applies to one Envoy instance and latency must stay near zero; use an external Rate Limit Service when the same quota must be shared across replicas or changed without reloading Envoy.

Decision and constraints

The choice depends on scope, latency budget, and how often the policy changes. This guide assumes Envoy 1.20 or newer, where the HTTP rate-limit filters described here are available.

• Scope. Local counters exist only in the Envoy process that receives the request. External counters live in a backing store and are visible to every Envoy instance.
• Latency budget. Local counting is in-memory. External counting requires a gRPC round-trip to the rate-limit service.
• Policy dynamism. Local limits require a configuration reload. External limits can be updated in the service and take effect immediately.

Option comparison

Dimension	Local in-process rate limit	External Rate Limit Service
Filter	envoy.filters.http.local_ratelimit	envoy.filters.http.ratelimit plus ratelimit_service
Counter storage	Per-process memory, token bucket	Central store, for example Redis via the Envoy Rate Limit Service
Typical latency impact	About 1-2 microseconds per request	About 1-3 milliseconds per request, depending on network
Cross-proxy coordination	None	Shared counters across proxies
Policy updates	Config reload required	Dynamic via service, no Envoy reload
Failure mode	No external dependency	Requires fail-open or fail-closed configuration

Trade-offs in practice

Local limiting works well for protecting a single upstream from bursts, or for per-tenant limits on a sidecar where that tenant lives on one pod. It cannot enforce a global quota such as 1,000 requests per minute across ten replicas, because each replica maintains its own bucket and the effective limit becomes roughly ten times higher.

External limiting fits API quotas, per-user or per-API-key limits, and multi-region consistency. The extra round-trip appears in p99 latency, and the rate-limit service is on the critical path. You must set failure_mode_allow to decide whether an outage blocks traffic (false) or lets it through (true).

Concrete configuration: local in-process rate limit

Run on a host where you can write the Envoy configuration file and reload the process. You need read/write access to the config directory and permission to signal Envoy or call its admin API. This example limits a route to 100 requests per 60 seconds with a burst of 150.

static_resources:
  listeners:
  - address:
      socket_address:
        address: 0.0.0.0
        port_value: 8080
    filter_chains:
    - filters:
      - name: envoy.filters.network.http_connection_manager
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
          stat_prefix: ingress_http
          http_filters:
          - name: envoy.filters.http.local_ratelimit
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
              stat_prefix: http_local_rate_limiter
              token_bucket:
                max_tokens: 150
                tokens_per_fill: 100
                fill_interval: 60s
              filter_enabled:
                runtime_key: local_rate_limit.enabled
                default_value:
                  value: true
              filter_enforced:
                runtime_key: local_rate_limit.enforced
                default_value:
                  value: true
              response_status_code: 429
          - name: envoy.filters.http.router
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router

Apply the file with envoy -c /path/to/envoy.yaml or reload through the admin API on port 9901. Keep the previous configuration file available so you can restart with it if the new limit blocks legitimate traffic.

Concrete configuration: external Rate Limit Service

This pattern requires a reachable Rate Limit Service. Place the filter before routing so the decision happens before the request reaches an upstream.

http_filters:
- name: envoy.filters.http.ratelimit
  typed_config:
    "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
    domain: example_domain
    rate_limit_service:
      grpc_service:
        envoy_grpc:
          cluster_name: ratelimit_cluster
      timeout: 0.5s
    failure_mode_allow: false

clusters:
- name: ratelimit_cluster
  connect_timeout: 0.25s
  type: STRICT_DNS
  lb_policy: ROUND_ROBIN
  load_assignment:
    cluster_name: ratelimit_cluster
    endpoints:
    - lb_endpoints:
      - endpoint:
          address:
            socket_address:
              address: <RATE_LIMIT_SERVICE_HOST>
              port_value: 8081

Add route-level descriptors in the route configuration to define what to limit, such as a header value or destination. The policy itself is stored in the Rate Limit Service, not in Envoy.

Validation

1. Start Envoy with the local filter configuration and confirm the process is listening on port 8080 and the admin port 9901.
2. Send a burst of test requests, for example 200 sequential calls to the test endpoint, and inspect the HTTP status codes.
3. Query Envoy stats for the local filter counter: curl -s http://localhost:9901/stats | grep http_local_rate_limiter. Expect the counter to increase as requests are consumed and 429 responses to appear once the bucket is empty.
4. For the external service, check ratelimit.ok, ratelimit.error, and ratelimit.over_limit in Envoy stats or Prometheus. Compare request latency with and without the external filter to confirm the added round-trip.

Run read-only stats queries without elevated privileges. Configuration changes require admin access. Do not run load tests against production endpoints without a maintenance window.

Limitations and operational checks

Local limits cannot share state across pods, so a configured limit is multiplied by replica count. External limits add a network dependency; monitor the rate-limit service for latency and error rate. If you change configuration, keep the previous version to roll back by restarting Envoy with the old file or reverting through the admin API.