Diagnosing and Resolving NGINX 502 Bad Gateway Errors

The Core Problem: The Broken Proxy Link

An HTTP 502 Bad Gateway error occurs when NGINX, acting as a reverse proxy, attempts to connect to a backend application server (the upstream) but receives an invalid response or no response at all. Unlike a 504 Gateway Timeout, which implies the backend is taking too long, a 502 typically indicates a connection failure or a protocol mismatch.

Quick Diagnostic Reference

Error Log Message	Likely Cause	Primary Check
connect() failed (111: Connection refused)	Upstream service is offline or listening on a different port.	Check process status and listening ports.
connect() failed (13: Permission denied)	OS-level security (SELinux) blocking the network request.	Check SELinux booleans.
upstream sent too big header	Response headers exceed NGINX buffer limits.	Increase proxy_buffer_size.
recv() failed (104: Connection reset by peer)	Backend crashed or closed the connection prematurely.	Check application logs for segmentation faults or crashes.

Step-by-Step Resolution Path

1. Verify Upstream Availability

Before modifying NGINX configurations, determine if the backend application is actually running and reachable from the NGINX server.

Run the following command on the server hosting NGINX (replace 127.0.0.1:8080 with your actual upstream address):

# Run as a user with network permissions
curl -I http://127.0.0.1:8080

• If it fails: The problem is the application, not NGINX. Check if the service is running (e.g., systemctl status my-app).
• If it succeeds: The backend is healthy; the issue lies in the communication between NGINX and the backend.

2. Check Port Binding

Ensure the application is listening on the exact port defined in your proxy_pass directive. Use the ss utility to verify the listening socket:

# Run as root or with sudo to see process names
ss -tulpn | grep :8080

Verify that the process name matches your expected application and that it is listening on the correct interface (e.g., 127.0.0.1 for local or 0.0.0.0 for all interfaces).

3. Resolve OS-Level Permission Blocks (SELinux)

On RHEL, CentOS, or Fedora systems, SELinux may prevent NGINX from initiating outbound network connections, resulting in a "Permission denied" error in the logs.

Check if the httpd_can_network_connect boolean is disabled:

# Run as root
getsebool httpd_can_network_connect

If it is off, enable it to allow NGINX to proxy requests:

# Run as root
sudo setsebool -P httpd_can_network_connect 1

Risk: This allows NGINX to connect to any network port. For stricter environments, use a targeted SELinux policy for specific ports.

4. Adjust Proxy Buffer Sizes

If your application sends large cookies or complex headers, NGINX may drop the connection if the response exceeds the default buffer size.

Add these directives to your location or server block in /etc/nginx/nginx.conf:

location / {
    proxy_pass http://backend_server;
    proxy_buffer_size 128k;
    proxy_buffers 4 256k;
    proxy_busy_buffers_size 256k;
}

Limitation: Increasing buffers consumes more memory per request. Monitor RAM usage under high load after applying these changes.

Applying Changes and Verification

To apply configuration changes without dropping active client connections, use the reload signal rather than a full restart:

# Run as root or sudo
nginx -t && nginx -s reload

The nginx -t command is critical; it validates the syntax before reloading. If the syntax is invalid, the reload will fail, leaving the previous working configuration active.

Escalation Criteria

If the 502 persists after these steps, escalate to the following areas:

• Network Infrastructure: Check firewalls (iptables/ufw) or cloud security groups between the NGINX server and the upstream server.
• Application Profiling: If the error is intermittent, check the backend for "Out of Memory" (OOM) kills or thread pool exhaustion.
• Protocol Mismatch: Verify if the backend expects HTTPS while NGINX is using proxy_pass http://....