NGINX 502 Bad Gateway Troubleshooting Guide

Step‑by‑step guide to identify why NGINX returns 502 errors, check logs, test connectivity, adjust timeouts and worker limits, and verify the fix.

Recognizable condition

Clients receive HTTP 502 Bad Gateway. In the NGINX error log you see messages such as:

upstream prematurely closed connection while reading response header from upstream
connect() failed (111: Connection refused) while connecting to upstream
upstream timed out (110: Connection timed out) while reading response header from upstream

Cause / diagnostic table

Possible cause	Typical log hint
Upstream server unresponsive or slow	‘upstream prematurely closed connection’ or ‘connect() failed’
Network packet loss or intermittent connectivity	Same as above, often with timeout values
Keepalive pool exhausted	‘keepalive timeout’ messages or ‘worker_connections are not enough’
Proxy read/send timeout too low	‘upstream timed out’ while waiting for response
Insufficient worker_processes / worker_connections for load	‘worker_connections are not enough’ or high idle CPU

Ordered checks

Review error.log
Locate the relevant NGINX error log (usually /var/log/nginx/error.log). Look for the upstream messages listed above. Note the timestamp and the upstream server address.
```
sudo tail -n 20 /var/log/nginx/error.log
```
Required permission: read access to the log file (typically root or nginx user).

Risk: none – read‑only.
Test direct connectivity to the upstream
From the NGINX host, try to reach the upstream server using curl or telnet on the same port NGINX uses.
```
# Replace with actual upstream host and port
curl -v http://upstream.example.com:8080/health
# or
 telnet upstream.example.com 8080
```
Permission: ability to open outbound TCP connections (usually unrestricted).

Expected check: you receive a valid HTTP response (200) or the TCP handshake succeeds. If the connection fails or is slow, the problem lies outside NGINX.

Risk: none – read‑only test.
Verify upstream health via NGINX status or external monitor
If you have the stub_status module enabled, fetch the status page:
```
curl http://localhost/nginx_status
```
Look for Active connections, Reading, Writing, Waiting values. High numbers of Waiting may indicate idle keepalive connections exhausting the pool.

Alternatively, check any third‑party health‑check tool you use.

Permission: read access to localhost NGINX status port (usually 80).
Examine proxy timeout directives
In the relevant location or upstream block, check values for:
- proxy_read_timeout
- proxy_send_timeout
- proxy_connect_timeout
Typical default is 60s. If your upstream application regularly takes longer, you will see timeout errors.
```
# Example snippet from /etc/nginx/nginx.conf or a site config
location /api/ {
    proxy_pass http://backend;
    proxy_read_timeout  120s;
    proxy_send_timeout  120s;
    proxy_connect_timeout 10s;
}
```
Permission: read the config file (root or nginx).
Check worker_processes and worker_connections
Open the main NGINX config (/etc/nginx/nginx.conf) and verify:
- worker_processes – often set to auto or a number matching CPU cores.
- worker_connections inside the events block – defines how many simultaneous connections each worker can handle.
Calculate max clients: worker_processes × worker_connections. If your traffic approaches this limit, you may see ‘worker_connections are not enough’.
```
events {
    worker_connections 4096;
}

http {
    # …
    worker_processes auto;
}
```
Permission: read config; changes require reload.

Fixes tied to findings

Increase proxy timeouts

If the error log shows timeout messages and direct upstream tests succeed but are slow, raise the timeout values:

location /api/ {
    proxy_pass http://backend;
    proxy_read_timeout  300s;   # increased from 60s
    proxy_send_timeout  300s;
    proxy_connect_timeout 30s;
}

Apply the change, then reload NGINX:

sudo nginx -t && sudo systemctl reload nginx

Risk: higher timeouts keep idle connections longer, consuming worker resources if the upstream is truly slow.

Tune keepalive connections

If logs indicate keepalive exhaustion, adjust the upstream keepalive settings:

upstream backend {
    server 10.0.0.10:8080;
    keepalive 32;          # number of idle keepalive connections per worker
    keepalive_timeout 60s;
}

Reload as above.

Risk: too many keepalive connections can exhaust file descriptors if worker_connections is low.

Adjust worker resources

When the error log shows ‘worker_connections are not enough’, increase either worker_connections or worker_processes:

events {
    worker_connections 8192;   # doubled from 4096
}

http {
    worker_processes 4;        # explicit, matches CPU cores
}

After editing, test config and reload:

sudo nginx -t && sudo systemctl reload nginx

Risk: raising worker_processes without matching worker_connections can lead to inefficient CPU use or file‑descriptor starvation.

Address network or upstream health

If direct connectivity checks fail or show high latency, involve the network or application team to:

Check for packet loss, firewall drops, or routing issues.
Verify the upstream service is healthy and not overloaded.
Consider enabling NGINX health checks (health_check) if using NGINX Plus or a compatible module.

Escalation criteria

Proceed to escalation when:

502 errors persist after adjusting timeouts, keepalive, and worker settings.
Direct upstream tests succeed with low latency, yet NGINX still logs timeouts.
Error log shows resource exhaustion messages such as ‘worker_connections are not enough’ or ‘open() #xx failed (24: Too many open files)’ despite config increases.
CPU, memory, or file‑descriptor usage on the NGINX host is near limits.

At this point, engage:

Application owners to investigate upstream response times.
Network engineers to examine connectivity and load‑balancer layers.
Consider adding a dedicated load‑balancer or caching layer if the upstream cannot be scaled.

Verification

After applying changes, confirm the fix with a controlled load test:

Reload NGINX and ensure nginx -t reports success.
Run a short benchmark (e.g., wrk -t12 -c400 -d30s http://your-site/) while watching the error log:

sudo tail -f /var/log/nginx/error.log

Look for absence of the upstream timeout or connection‑failed messages.

Additionally, confirm that direct requests to the upstream return expected responses with latency below the new timeout:

time curl -o /dev/null -s -w "%{time_total}\n" http://upstream.example.com:8080/health

If latency remains close to or exceeds the timeout, consider further tuning or upstream optimization.

Rollback (if needed)

Because configuration changes alter system state, keep a backup of the edited file(s). To revert:

Restore the original config from backup or version control.
Run sudo nginx -t to validate.
Reload NGINX: sudo systemctl reload nginx.

Verify that the error log returns to its previous state and that the 502 error pattern reappears, confirming the rollback restored the prior behavior.

Limitations

This guide assumes:

NGINX is serving as a reverse proxy or load balancer.
You have access to error logs and the ability to reload the service.
The upstream service is reachable from the NGINX host (no intermediate NAT interfering with tests).
No custom third‑party modules interfere with standard timeout or health‑check directives.

If any of these conditions do not hold, additional steps may be required.

Diagnosing NGINX 502 Bad Gateway Caused by Upstream Timeout or Misconfiguration