What causes intermittent HikariCP connection pool exhaustion under peak load?

Direct Answer

The most reliable early indicator of impending pool exhaustion is the pending thread count (HikariCP metric hikaricp_pending_threads or JMX PendingThreads), not active connections alone. When pending threads rise above zero and stay there, requests are queuing for connections before the connectionTimeout fires.

For maximumPoolSize, start with min(2 * CPU_cores, DB_max_connections - headroom) where headroom reserves 10–20% of the database's max_connections for administrative sessions and other applications. Validate against the database's actual process list, not just the configured limit.

Confirmed Behavior vs. Likely Explanation

Confirmed (version-agnostic)

• HikariCP reports ActiveConnections = maximumPoolSize and IdleConnections ≈ 0 when the pool is saturated.
• The connectionTimeout (default 30 s) only triggers after a thread has waited that long; PendingThreads captures the queue depth before the timeout.
• Connection leaks (missing close() in finally or try-with-resources) monotonically reduce available connections regardless of pool size.

Likely in Your Scenario

• Peak traffic spikes push concurrent request count above maximumPoolSize for longer than the database's query latency, so threads accumulate in the pending queue.
• Database CPU appears low because sessions are waiting on locks, I/O, or network round-trips—not because the DB has spare capacity.
• Spring Boot 2.x (HikariCP 3.x) defaults to maximumPoolSize=10; Spring Boot 3.x (HikariCP 5.x) keeps the same default but changes leakDetectionThreshold default to 0 (disabled).

Minimal Diagnostic Steps

1. Enable pending-thread metrics: Add Micrometer/Prometheus exposition for hikaricp_pending_threads and hikaricp_active_connections. Alert when pending_threads > 0 for > 10 s.
2. Cross-check with DB: Run SELECT count(*) FROM pg_stat_activity WHERE state = 'active'; (PostgreSQL) or SHOW PROCESSLIST (MySQL) during a spike. If DB active sessions < application ActiveConnections, you have leaked or orphaned connections.
3. Turn on leak detection: Set spring.datasource.hikari.leakDetectionThreshold=60000 (60 s) in application.yml for one release cycle. Logs will show stack traces of leaked checkouts.
4. Calculate target pool size: target = min(2 * availableProcessors(), db_max_connections * 0.8 / app_instance_count). Apply, load-test, and observe pending-thread metric.

One Missing Detail That Changes the Recommendation

Are you running multiple application instances behind a load balancer? If yes, divide the database's usable connection budget by the instance count before applying the formula above. If no, the single-instance formula stands.