nginx limit_req burst behavior: queued requests with nodelay vs without nodelay

Burst and Nodelay Interaction

In Nginx, the burst parameter defines a "bucket" of slots that can be filled when requests arrive faster than the defined rate. Whether you use nodelay or not, every request that exceeds the rate but fits within the burst capacity consumes one burst slot.

Without nodelay

Requests that exceed the rate are placed in a queue. Nginx processes these queued requests strictly at the defined rate (e.g., 10 r/s). This introduces artificial latency for the client, as the request waits in the queue before being passed to the backend.

With nodelay

Requests that exceed the rate but fit within the burst capacity are processed immediately. However, they still occupy a slot in the burst bucket. These slots are only freed at the defined rate. If a client sends a burst of 20 requests instantly, they are all processed immediately, but the bucket is now full. Any subsequent request arriving before the rate-based "refill" occurs will be rejected with a 503 error.

Handling Load Balancers and Proxy IPs

The binary_remote_addr key tracks the immediate connection source. If Nginx is behind a load balancer, this will be the proxy's IP, causing all clients to share a single rate limit bucket.

• Recommendation: Use the real_ip module to extract the client IP from the X-Forwarded-For header. Once configured, binary_remote_addr will correctly reflect the actual client IP.
• Alternative: Use $http_x_forwarded_for as the key in limit_req_zone, but be aware that this header can be spoofed unless your proxy explicitly overwrites it.

Sizing the Shared Memory Zone

The memory zone size depends on the key used. For binary_remote_addr, Nginx uses a compact binary representation of the IP address.

Zone Size	Approx. Client IPs (binary_remote_addr)
1MB	~16,000
10MB	~160,000

For thousands of unique IPs, a 10MB zone is generally sufficient and safe. If you use a string-based key like $http_x_forwarded_for, memory consumption will be significantly higher per entry.

Verification Steps

To verify the behavior, use a benchmarking tool like wrk or ab:

# Test for 503s and latency with a burst of requests
wrk -t2 -c10 -d5s http://your-server/api

Check the Nginx error log for limiting requests messages to confirm when the burst capacity is exhausted.