Too many connections error in Vault database secrets engine connection pool

Diagnosing intermittent connection pool exhaustion

When Vault’s database secrets engine hands out dynamic credentials, each role creates its own connection pool governed by max_open_connections, max_idle_connections, and connection_max_lifetime. In environments with multiple roles targeting the same database, the sum of per‑role pools can exceed the database’s max_connections limit. This manifests as the database reporting too many connections or the Vault client returning a “connection pool exhausted” error. The problem is often intermittent, surfacing during bursty workloads or when applications fail to close connections before a lease expires.

Key constraints include:

• Per‑role pools are not coordinated; each max_open_connections value adds independently.
• The connection_max_lifetime setting may clash with the database’s idle timeout, leading to stale connections.
• Lease revocation and periodic cleanup share the same pool, adding contention.

To resolve the issue, we need to understand how these knobs interact and whether additional isolation or metrics can expose the root cause.

• What metrics or Prometheus labels best capture the real‑time saturation of per‑role pools?
• How can we configure max_open_connections per role so the aggregate stays below the database’s max_connections without manual tuning for each burst?
• Does setting a non‑zero connection_max_lifetime consistently prevent stale connections from causing transient “too many connections” errors?