Memory growth in ngrok HTTP tunnel connection pool under sustained load

Answer first

There is no single NGROK_POOL_IDLE_TIMEOUT value that universally limits memory growth without adding latency. The trade-off depends on request arrival pattern, upstream keep-alive settings, and whether the inspect API is retaining request data. Lowering the idle timeout reduces the number of idle upstream sockets that ngrok keeps open, which can reduce resident memory, but it also increases connection churn and first-byte latency for bursts of traffic. The inspect API request storage is independent of the pool idle timeout and can dominate growth if enabled.

Confirmed facts vs likely explanation

Confirmed behavior, with version assumptions:

• ngrok’s tunnel process maintains a pool of upstream connections to the local server. Pool size can grow under sustained traffic if connections are not returned to the pool or if keep-alive timeouts are long.
• The pool idle timeout is referenced in community reports as the undocumented environment variable NGROK_POOL_IDLE_TIMEOUT. Default behavior is not officially documented and may change between releases.
• Enabling the inspect API adds additional request retention for debugging. That retention is separate from connection pooling and can exacerbate memory growth.

Likely explanation, marked for review:

• Linear memory growth under sustained load is commonly attributed to idle connections holding buffers, and to request objects retained by the inspect API. A leak in keep-alive logic or in the Go runtime’s connection handling has been reported anecdotally, but this is not confirmed as a universal defect.

Minimal steps for this case

1. Confirm whether inspect API is enabled. If yes, test with it disabled to isolate request retention from pool memory.
2. Enable verbose logging and observe connection lifecycle: ngrok --log-level=debug. Look for repeated connection creation without matching close events.
3. Monitor active connections over time via ngrok’s local API, e.g., GET http://localhost:4040/api/tunnels, and correlate with process RSS.
4. If growth stabilizes after idle periods, consider lowering the pool idle timeout via NGROK_POOL_IDLE_TIMEOUT only after testing latency impact in your traffic pattern. Do not set extremely short values.
5. Upgrade to the latest stable ngrok release and re-test, as pool-related fixes have been released in the past.

One missing diagnostic detail that changes the recommendation: ngrok version and whether inspect API is enabled with request body retention. Without that, guidance on timeout tuning remains speculative.