Application Connection Pool Interaction with Linux TCP TIME_WAIT Reuse Under High Connection Churn
29.5K reputation · 25 Sept 2020, 14:37 UTC
Goal: Determine whether enabling the kernel option net.ipv4.tcp_tw_reuse alleviates intermittent connection‑pool exhaustion observed in high‑churn TCP workloads when the application pool evicts idle connections based on a timeout.
Constraints: The application’s connection pool maintains a maximum size and evicts idle sockets after a configurable idle‑timeout, leaving sockets in the ESTABLISHED state until the pool limit is reached. Enabling tcp_tw_reuse may allow the kernel to reuse TIME_WAIT sockets for new outbound connections, but it is only safe for protocols that tolerate relaxed TCP ordering and may affect reliability if delayed packets are accepted.
Uncertainty: It is unclear how the interaction between the pool’s idle‑timeout eviction and the kernel’s TIME_WAIT reuse influences the steady‑state counts of ESTABLISHED and TIME_WAIT sockets, and whether the observed exhaustion symptoms are primarily driven by port‑allocation limits or file‑descriptor consumption.
Does enabling net.ipv4.tcp_tw_reuse reduce the frequency of bind() errors due to ephemeral‑port exhaustion without causing a corresponding rise in ESTABLISHED sockets held by the pool?
Given a fixed pool max size and idle‑timeout, what is the threshold of connection rate at which TIME_WAIT reuse begins to offset pool‑exhaustion symptoms, and how does this threshold shift when tcp_fin_timeout is tuned?