keepalive versus worker_connections: trade‑off for short‑lived connection latency in nginx

Answer the question first

For short‑lived, bursty traffic the keepalive_timeout is usually the more effective lever for reducing per‑request latency. A higher timeout keeps the TCP socket open, eliminating the TCP three‑way handshake for subsequent requests from the same client. worker_connections is a safety net: raise it only if you hit the FD ceiling and the keep‑alive pool is already saturated.

Typical rule of thumb

• Keep keepalive_timeout between 60 s and 120 s if the same client IP makes many rapid requests (e.g., API polling or mobile app traffic). This keeps enough sockets alive to cover the burst without holding them indefinitely.
• Set keepalive_requests to the expected maximum number of requests per burst (default 100 is usually fine). Each keep‑alive socket can serve that many requests before being closed.
• Do not raise worker_connections above the OS file‑descriptor limit (ulimit -n) unless you have verified that memory consumption remains acceptable.
• Ensure keepalive_timeout is lower than or equal to the upstream server’s timeout (e.g., proxy_read_timeout) so that nginx does not keep a socket open for a connection that the upstream will already have dropped.

Why this works

In nginx 1.10+ each keep‑alive connection consumes one worker_connections slot. The total number of simultaneous keep‑alive sockets is bounded by worker_connections * (1 + (keepalive_requests‑1)/keepalive_requests). Raising keepalive_timeout increases the number of sockets that stay open, potentially approaching that bound. If the bound is reached, nginx starts closing idle sockets or rejecting new ones, which forces a fresh TCP handshake and increases latency.

Practical steps for your environment

1. Check current limits:

   ulimit -n
   cat /proc/sys/fs/file-max
   

2. Measure current keep‑alive usage:

   sudo lsof -p $(pidof nginx) | wc -l
   # or use stub_status: http://localhost/nginx_status
   

3. If the number of open sockets is close to worker_connections (e.g., >90% of the limit), consider raising worker_connections after increasing the OS limit.
4. Otherwise, tune keepalive_timeout:

   http {
       keepalive_timeout 90s;
       keepalive_requests 100;
       # upstream timeouts
       proxy_read_timeout 120s;
       proxy_connect_timeout 30s;
   }
   

5. Run a short load test (wrk or ab) that simulates many short requests from the same IP and verify that average latency drops and no 503s appear.

Interaction with upstream timeouts

When nginx keeps a socket alive longer than the upstream’s timeout, the upstream may close the connection silently. nginx will then detect a broken socket on the next request and open a new one, adding latency. Therefore set keepalive_timeout to a value that is no greater than the smallest upstream timeout you rely on.

Missing diagnostic detail

To fine‑tune the recommendation I’d need to know the average number of distinct client IPs per second and the typical burst size per client. If you can share those numbers, I can suggest a more precise keepalive_timeout and whether a worker_connections increase is warranted.