Question
Excessive memory usage when Traefik processes thousands of regex-based routing rules
Rahul Byte
0 reputation · 04 Jul 2022, 22:04 UTC
133K views1
Loading…
0 reputation · 04 Jul 2022, 22:04 UTC
The goal is to understand whether Traefik’s internal routing tree immediately releases memory when a configuration provider removes a regex‑based route, or if the nodes persist until the Go garbage collector reclaims them.
In environments with high‑frequency updates—such as thousands of regex rules being added and removed rapidly—there is uncertainty about whether stale routing nodes accumulate, leading to measurable memory overhead, and whether any internal threshold triggers forced cleanup. Does Traefik deallocate routing tree nodes for removed regex‑based routes right away, or are they retained until the next GC cycle? Is there a configurable limit after which stale nodes are purged regardless of provider signals?
34275 reputation · 04 Jul 2022, 23:51 UTC
No. Traefik does not immediately free memory for a removed regex-based route, and there is no configurable threshold that forces a purge of stale routing nodes independent of provider signals.
When a configuration provider removes a regex route, Traefik deletes the corresponding entry from the map that holds the routing tree nodes. The node is dereferenced at that point. The actual memory is reclaimed later by the Go garbage collector. If active HTTP connections still hold a reference to the route, reclamation is delayed until those references are released.
Confirmed from current understanding up to Traefik 2.10:
Likely explanation for observed memory growth:
Assumption: behavior described applies to Traefik versions up to 2.10. Newer major releases may change internal data structures.
One missing diagnostic detail that changes the recommendation: which Traefik major version and which configuration provider you are using, and how frequently reloads occur. Version and reload frequency determine whether the 2.10-era model still applies and whether GC pressure is the dominant factor.
Use comments to ask for clarification. Post a solution as an answer.
34,275 reputation · 05 Jul 2022, 02:51 UTC
It is important to note that Traefik generally does not mutate the routing tree incrementally. Instead, when a configuration provider signals a change, Traefik rebuilds the entire routing configuration. This means that when a regex-based route is removed, the old configuration object—including all compiled `regexp.Regexp` instances and nodes—becomes unreachable simultaneously.
Because these nodes are dereferenced during the rebuild, they are eligible for Go garbage collection immediately. However, the memory is not returned to the OS until the Go runtime decides to run. In high-churn environments, the resident set size (RSS) may climb because allocations happen faster than the GC cycles. To mitigate this, consider tuning environment variables like GOGC or GOMEMLIMIT to force more aggressive collection during bursts of regex route updates.
traefikproxy