SIGSEGV in worker process after prolonged HTTP/1.1 keepalive traffic

Upstream Keepalive and SIGSEGV

Updating to nginx ≥ 1.19.8 resolves the specific upstream keepalive memory leak that previously led to heap corruption and subsequent SIGSEGV (segmentation fault) in worker processes. This issue typically manifested during prolonged HTTP/1.1 traffic where connections were held open for extended periods, causing a slow accumulation of unreleased memory that eventually corrupted the process address space.

Verification of the Fix

To verify that the leak is resolved in your environment, monitor the Resident Set Size (RSS) of the worker processes under a sustained load test simulating long-lived keepalive connections. A resolved leak will show memory usage stabilizing (plateauing) rather than increasing linearly over time.

# Monitor worker memory usage every 5 seconds
watch -n 5 "ps aux | grep nginx | grep worker"

Worker Processes and CPU Affinity

Regarding the interaction between worker_processes auto; and worker_process_auto_affinity on;:

Balanced Placement and Heterogeneous Cores

Enabling worker_process_auto_affinity does not guarantee balanced worker placement across different core types (such as ARM big.LITTLE or NUMA nodes). The auto setting simply spawns a number of workers equal to the total number of available CPU cores. The affinity mechanism then attempts to bind each worker to a specific core to reduce cache misses and context switching.

On heterogeneous architectures, the OS scheduler and Nginx's basic affinity logic may not distinguish between "performance" and "efficiency" cores. This can result in some workers being pinned to slower cores while others handle the bulk of the traffic, leading to uneven load distribution and suboptimal throughput.

Recommendation for big.LITTLE/NUMA Systems

To avoid suboptimal performance on heterogeneous systems, explicit configuration is required. You should manually define the worker count and use worker_cpu_affinity to map workers specifically to your high-performance cores.

Example configuration for a system with 4 performance cores:

worker_processes 4;
worker_cpu_affinity 0001 0010 0100 1000;

If you are using a NUMA architecture, ensure that the worker_cpu_affinity mask aligns with the physical memory nodes to prevent expensive cross-node memory access.

Diagnostic Requirement

To provide a more precise affinity mask, please specify the exact CPU topology (e.g., number of performance vs. efficiency cores) of your hardware.