Intermittent connection pool exhaustion: idle connection drop vs request queuing in xd

Answer

Under a transient spike xd does not proactively drop idle connections to free capacity. When the pool is at its configured maximum size and no idle connections are available, incoming checkout requests are queued. Idle connections are not evicted to serve the spike.

If the pool is configured with fail-fast, requests wait up to the configured maximum wait time and then fail immediately without indefinite queuing. Leak detection does not change the queue-vs-drop decision; it can only free capacity indirectly by closing connections held past a threshold.

Confirmed behavior

• Pool reaches max size → new checkouts queue rather than trigger immediate idle eviction.
• Idle eviction occurs only on periodic idle-timeout checks or when a validation query fails on checkout/return.
• Fail-fast mode: requests exceed max wait time and fail, no queuing.
• Leak detection tracks checkout timestamps and may close a connection held beyond its threshold, freeing a slot for queued requests.

Likely explanation for intermittent exhaustion

The spike is absorbed by the request queue while the pool stays at max. Capacity is only freed when an in-use connection is returned, or later when idle-timeout / validation removes truly idle connections. If max wait is short or fail-fast is enabled, clients see intermittent failures during the spike rather than silent drops.

Steps needed for this case

1. Enable detailed pool logging and reproduce the spike. Confirm queue entries appear before any eviction logs.
2. Record current settings: maxPoolSize, maxWaitMillis, idleTimeoutMinutes, validationQuery enabled, leakDetectionThreshold.
3. Adjust idle timeout to a short test value and repeat the spike to observe if idle eviction frees slots for queued requests.
4. With leak detection enabled, hold a connection beyond the leak threshold and verify pool logs a leak close and a queued request is then served.

Assumption: behavior described applies to the standard xd connection pool implementation. Exact timing depends on configured idle timeout and validation settings, which must be known to predict spike handling. Version differences may introduce aggressive eviction policies.

One missing diagnostic detail that changes the recommendation: what is the current maxWait / fail-fast configuration and idleTimeout value in your xd deployment? The queue depth and failure mode depend directly on those.