How to Diagnose Intermittent Connection Pool Exhaustion in GitHub Actions?

Short answer

Intermittent exhaustion is almost always a symptom of a temporary spike in connection demand that outpaces the pool’s configured limits or a leak that only becomes apparent under load. The most common culprits are:

• Inadequate maxPoolSize for the number of concurrent jobs or threads the workflow spawns.
• Connections that are left open across job boundaries because the application keeps a shared pool between steps.
• Transient network hiccups that leave connections in a “broken” state, preventing the pool from recycling them quickly.

Confirmed facts from your description

• The workflow runs on GitHub‑hosted Ubuntu runners.
• Parallel jobs are below the database’s advertised max pool size.
• Failures occur sporadically, not on every run.

Diagnostic steps you can take inside the workflow

1. Enable pool debug logging
   

   Most JDBC drivers expose a debug level that prints pool state changes. For HikariCP, add the following to your application.yml or set the environment variable before the test step:

   export HIKARI_DEBUG=true
   # or in Java
   System.setProperty("com.zaxxer.hikari.pool.PoolBase", "DEBUG");
   

   GitHub Actions will capture the log output automatically. Look for lines such as HikariPool-1 - Created new connection and HikariPool-1 - Connection returned to see how many connections are in use.

2. Query PostgreSQL for live connections
   

   After each step that opens a connection, run a lightweight query against pg_stat_activity to count active sessions:

   echo "SELECT count(*) FROM pg_stat_activity WHERE datname = 'your_db';" | psql -h $DB_HOST -U $DB_USER -d $DB_NAME -t
   

   Store the output in a file and compare a successful run to a failing one.

3. Introduce a short pause between jobs
   

   Insert a sleep 10 step after the test job to give the pool time to recycle connections before the next job starts. If the failure rate drops, a timing issue is likely.

4. Temporarily raise the pool size
   

   Set maxPoolSize to a value well above the expected peak (e.g., 200) and rerun the workflow. If the exhaustion events disappear, the pool size was the bottleneck.

5. Check runner resource limits
   

   GitHub‑hosted runners expose CPU and memory limits. Use cat /proc/meminfo and cat /proc/cpuinfo to verify that the runner isn’t hitting its quota during the test run. If the runner is throttled, consider a larger runner size (e.g., ubuntu-22.04-large) or a self‑hosted runner in a dedicated environment.

Configuration changes to reduce exhaustion risk

• Set maxPoolSize to at least the sum of connections needed by all parallel jobs plus a safety margin.
• Configure idleTimeout and maxLifetime to ensure idle connections are closed and refreshed regularly.
• If using a shared pool across steps, explicitly close the pool at the end of each job: DataSource ds = ...; ds.close();
• Enable connectionTestQuery or validationTimeout so the driver can detect broken connections early.
• Use environment variables to enforce the same pool settings across all job contexts.

Should you switch runners?

Only move to a self‑hosted or larger runner if:

• All diagnostic steps above still show exhaustion.
• The database connection limit is hit frequently even when the pool is oversized.
• Network latency or packet loss between GitHub runners and the database is proven (e.g., via repeated ping or traceroute tests).

Otherwise, tuning the pool and ensuring proper cleanup is usually enough.

Missing detail that changes the recommendation

What is the maxPoolSize (or equivalent) configured in your application? Knowing the exact value will let us decide whether the pool is truly undersized or if the issue is elsewhere.