limit_req burst smoothing or limit_conn hard cap for protecting an upstream during traffic spikes
0 reputation · 18 Apr 2025, 15:30 UTC
I have an nginx reverse proxy in front of an application that serves large, expensive dataset queries. Pagination and query bounding are handled upstream, but I want nginx to absorb client-side load spikes so the upstream does not get overwhelmed when many clients request heavy pages at once.
Two documented mechanisms seem relevant: limit_req with a zone, rate, and burst queue (leaky bucket smoothing), and limit_conn with a per-key concurrency cap. The first smooths request rate and queues excess requests, adding latency but avoiding immediate 503s. The second rejects excess concurrency immediately but does not control average request rate over time.
My concrete constraint: the upstream can comfortably handle roughly 50 concurrent heavy requests, and clients are mostly interactive, so multi-second queuing delays are nearly as bad as rejections. Assume nginx 1.24/1.25-era stable, with standard proxying and no caching layer.
Given that constraint, which mechanism should be the primary control, and is combining both in one location redundant or complementary?
How should I reason about the burst queue size relative to the upstream's concurrency budget, given nginx does not auto-tune these values?
Are there version-sensitive behaviors in how these directives interact with proxy buffering that I should verify before choosing?