Diagnosing and Resolving Connection Pool Exhaustion in Dropwizard

The Symptom: Latency Spikes and Connection Timeouts

When a Dropwizard application suffers from connection pool exhaustion, the failure is rarely immediate. It typically manifests as a gradual increase in response latency, followed by a sudden surge of Timeout waiting for connection errors in the application logs. At this point, the Jetty worker threads are blocked waiting for a database connection that never becomes available, eventually leading to a complete service hang.

Diagnostic Quick-Reference

Observation	Likely Cause	Primary Metric to Check
Steady climb in active connections until max is hit	Connection Leak (Unclosed sessions)	db.pool.ActiveConnections
Spikes in wait time during peak traffic only	Under-provisioned Pool Size	db.pool.WaitCount
Low active connections but high request latency	Long-running Transactions/Locks	Database Slow Query Log

Step-by-Step Diagnostic Workflow

Follow these checks in order to isolate whether the issue is a configuration mismatch, a code-level leak, or a database performance bottleneck.

1. Analyze the Metrics Registry

Dropwizard exposes internal pool statistics via the /metrics endpoint. Access this endpoint (usually on the admin port) and look for the database pool gauges. If you are using the default HikariCP or Tomcat JDBC integration, look for the following indicators:

• ActiveConnections: If this value stays at the maxSize limit even during low traffic, you have a connection leak.
• WaitCount: A high number of threads waiting for a connection indicates that the pool is too small for the current concurrency level.

2. Audit the YAML Configuration

Verify your config.yml to ensure the pool is sized appropriately for your Jetty thread pool. A common mistake is setting a database pool significantly smaller than the number of Jetty worker threads, creating a bottleneck.

database: 
  driverClass: org.postgresql.Driver
  user: dbuser
  password: dbpassword
  url: jdbc:postgresql://db-host:5432/mydb
  # Connection Pool Settings
  properties: 
    maxSize: 32
    minSize: 8
    maxWaitMillis: 5000

Risk: Do not increase maxSize beyond the max_connections limit configured on your database server. Doing so will shift the failure from the application pool to the database server, potentially crashing the DB instance.

3. Trace Connection Lease Cycles

If metrics suggest a leak, enable DEBUG logging for the connection pool provider in your logging configuration. This allows you to see exactly when a connection is borrowed and when it is returned.

• For HikariCP: Set com.zaxxer.hikari to DEBUG.
• For Tomcat JDBC: Set org.apache.tomcat.jdbc.pool to DEBUG.

Search the logs for "leased" events that do not have a corresponding "returned" or "closed" event within a reasonable timeframe.

Applying the Fix

Scenario A: Fixing Connection Leaks

If you find that connections are not returning to the pool, ensure all JDBI handles or Hibernate sessions are wrapped in try-with-resources blocks. In Dropwizard/JDBI, failing to close a Handle is the most common cause of leaks.

// INCORRECT: Handle may remain open if an exception occurs
Handle handle = jdbi.open();
handle.execute("INSERT INTO users...");
handle.close();

// CORRECT: Auto-closeable ensures return to pool
try (Handle handle = jdbi.open()) {
    handle.execute("INSERT INTO users...");
}

Scenario B: Optimizing Pool Sizing

If the WaitCount is high but ActiveConnections fluctuates, increase the maxSize in your YAML. As a rule of thumb, start with (2 * CPU cores) + effective_spindle_count and adjust based on load test results using a tool like JMeter.

Scenario C: Reducing Transaction Hold Time

If connections are held too long, audit your code for "External API calls inside transactions." If a thread makes an HTTP request to another service while holding a DB connection, that connection is idle but unavailable to others.

• Fix: Move the API call outside the @Transaction block or the JDBI inTransaction lambda.

Verification and Rollback

To verify the fix, deploy the change to a staging environment and simulate peak load. Monitor the /metrics endpoint; the WaitCount should remain near zero, and ActiveConnections should drop back to minSize after the load ceases.

Rollback: If increasing the pool size causes the database server to reject connections (Too many connections error), revert the maxSize in the YAML configuration to the previous value and restart the service.