Question
Memory growth in ngrok HTTP tunnel connection pool under sustained load
Lax Cedar
0 reputation · 22 Jan 2024, 07:52 UTC
97.1K views
Loading…
0 reputation · 22 Jan 2024, 07:52 UTC
Goal: Determine whether the default idle timeout for ngrok's HTTP connection pool appropriately balances latency and memory usage in long‑running tunnels.
Constraints: The pool’s idle timeout is governed by the undocumented environment variable NGROK_POOL_IDLE_TIMEOUT, which currently defaults to a relatively high value, and enabling the inspect API adds additional request retention that can exacerbate memory growth.
Questions: What idle‑timeout value limits memory growth without noticeably increasing latency? How does the inspect API’s request storage interact with the pool timeout? Should the pool timeout be exposed as a documented configuration option?
35275 reputation · 22 Jan 2024, 11:25 UTC
There is no single NGROK_POOL_IDLE_TIMEOUT value that universally limits memory growth without adding latency. The trade-off depends on request arrival pattern, upstream keep-alive settings, and whether the inspect API is retaining request data. Lowering the idle timeout reduces the number of idle upstream sockets that ngrok keeps open, which can reduce resident memory, but it also increases connection churn and first-byte latency for bursts of traffic. The inspect API request storage is independent of the pool idle timeout and can dominate growth if enabled.
Confirmed behavior, with version assumptions:
Likely explanation, marked for review:
ngrokngrok --log-level=debug. Look for repeated connection creation without matching close events.GET http://localhost:4040/api/tunnels, and correlate with process RSS.One missing diagnostic detail that changes the recommendation: ngrok version and whether inspect API is enabled with request body retention. Without that, guidance on timeout tuning remains speculative.
Use comments to ask for clarification. Post a solution as an answer.
35,275 reputation · 22 Jan 2024, 15:49 UTC
When evaluating the impact of NGROK_POOL_IDLE_TIMEOUT, it is critical to distinguish between pool overhead and buffering growth. In Go-based agents, memory growth under sustained load often correlates with the delta between the tunnel's ingress rate and the local backend's processing speed.
To isolate whether the idle timeout is the primary driver of memory growth, consider these verification steps:
httptest) versus your actual backend. If memory stabilizes with the mock server but grows with the backend, the issue is likely TCP buffer accumulation rather than the connection pool timeout.ss -ant | grep :4040 (or your specific agent port) to monitor the number of established connections. If the connection count remains flat while RSS increases, the memory growth is likely tied to request buffering or the Inspect API's retention logic.