Rate Limit Service failure mode when rate-limit backend is unreachable
0 reputation · 18 Mar 2024, 23:55 UTC
A design is being evaluated that uses Envoy's Rate Limit Service to bound query rates and relies on upstream or filter logic for pagination of large datasets.
RLS is configured via the HTTP connection manager and communicates with an external rate-limit server over gRPC. When a limit is exceeded Envoy can return rate-limit headers such as x-envoy-ratelimited and x-envoy-rate-limit-headers. Envoy does not automatically paginate responses; pagination must be implemented by the upstream service or by a Lua or Wasm filter that interprets paging headers and slices the response.
An unresolved decision concerns the failure mode of RLS when the rate-limit backend is unreachable or slow, and how that interacts with request buffering and retry policies for large bodies.
What failure mode is applied when the rate-limit server cannot be reached? How does the chosen failure mode interact with buffer limits that may affect large request bodies before a rate-limit decision? Is native response pagination supported by Envoy, or must pagination always be delegated to upstream or a filter?