Question
Knex connection pool limits: when does raising max stop helping?
Ira River
0 reputation · 24 Oct 2020, 10:27 UTC
59.9K views1
Loading…
0 reputation · 24 Oct 2020, 10:27 UTC
Knex delegates connection pooling to tarn.js, with documented defaults of min 2 and max 10 connections per instance. Once the pool is exhausted, further queries queue waiting for a free connection, and transactions hold a dedicated connection for their entire lifetime.
My goal is to decide whether a perceived throughput bottleneck should be addressed by increasing pool max, or whether that just shifts the problem. Knex exposes pool introspection such as numUsed, numFree, and numPendingAcquires, so pending acquires can be measured before changing any settings. But pool metrics only show client-side queuing; they say nothing about slow SQL, missing indexes, or lock contention inside the database.
There is also a per-instance wrinkle: pool settings apply per Knex instance, so multiple instances against one database multiply total connections and can hit server-side limits such as PostgreSQL max_connections.
Assuming a recent Knex version (defaults may differ across majors):
34775 reputation · 24 Oct 2020, 20:50 UTC
Raising the Knex pool max stops helping the moment the bottleneck shifts from client-side pool exhaustion to the database server's own limits or query performance. If queries are already slow, the database is CPU/IO bound, or the server's max_connections is near, adding connections only increases contention and can degrade throughput.
Monitor numPendingAcquires alongside query duration. A sustained pattern where pending acquires are consistently above zero and active connections sit at max and query latency is normal (e.g., p95 under 50 ms) suggests pool exhaustion is the real bottleneck. In that case, raising max may help. If query duration is already high or the database shows CPU/IO saturation, raising max will not help—fix the query or add database capacity first.
A practical rule: if pending acquires exceed roughly 10% of total queries over a 5-minute window and the DB server has headroom, consider a modest increase (e.g., 10 → 20). Otherwise, investigate server-side metrics first.
Each Knex instance creates its own pool, so total connections = sum of all instance max values. Your combined max must stay below the database's max_connections, leaving room for administrative connections and other clients. For PostgreSQL, check SHOW max_connections;; for MySQL, SHOW VARIABLES LIKE 'max_connections';. If you have 4 instances and a server limit of 100, each instance's max should be around 20, but also subtract any other applications. Never set max above the server limit—you'll get connection errors instead of queuing.
Pool metrics like numUsed show a connection is occupied but do not reveal why. A long transaction and a slow query look identical from the client side. To distinguish, inspect the database's active queries—PostgreSQL's pg_stat_activity or MySQL's SHOW PROCESSLIST—and look for transaction start times and lock waits. Pool metrics alone are insufficient.
To give a precise recommendation, I need your database server's max_connections and current connection usage. If that limit is already close, raising max will not help and may cause errors. If it's far away, pool exhaustion is more likely the issue.
knexjsknex.client.pool).numPendingAcquires and numUsed over 10 minutes.max is justified.Use comments to ask for clarification. Post a solution as an answer.
34,775 reputation · 24 Oct 2020, 18:03 UTC
When the Knex pool’s acquireTimeoutMillis (default 60 000 ms) is exceeded, tarn.js increments numPendingAcquires and then rejects the acquire with a timeout error. This means a high pending‑acquire count can stem from the timeout itself rather than from the pool being truly exhausted. Checking the pool’s acquireTimeoutMillis value and looking for acquire‑timeout errors in your logs helps distinguish a genuine saturation from a configuration‑induced queue.