Diagnosing Goroutine Leaks in Go Services: From Rising Counts to Root Cause

Your Go service's goroutine count climbs steadily under load and never comes back down, even after traffic stops. Memory creeps up with it because every parked goroutine retains its stack. That is a goroutine leak, and the fix is almost never "restart more often" — it is finding the blocking call that never returns. This guide walks through recognizing the condition, localizing the cause with pprof, and applying fixes matched to what you find. It assumes Go 1.18+ and access to net/http/pprof or runtime/pprof.

Recognizing a leak versus normal growth

A rising goroutine count during warmup or a traffic spike is normal. The leak signature is specific: runtime.NumGoroutine() (or the go_goroutines Prometheus metric) grows monotonically under steady load and does not return to baseline several minutes after load is removed. Confirm this before investigating — chasing a warmup curve wastes time.

Common causes at a glance

Symptom in profile	Likely cause	Typical fix
Stacks parked in chan receive/send	No receiver/sender, or quit channel never signaled	Close producer channels; select on ctx.Done()
Stacks in select with no cancellation case	Context not propagated into long-running work	Pass context.Context through; select on ctx.Done()
Stacks in net/http response reading	resp.Body never closed	defer resp.Body.Close() after error check
Growing timer-related stacks	time.After in a hot loop; ticker never stopped	Use time.NewTimer + Stop(), or ticker.Stop()
Identical worker-pool stacks accumulating	Pool shutdown path never triggered	Signal and drain the quit channel on shutdown

Ordered checks

1. Confirm the leak. Watch go_goroutines under constant load, then stop traffic. If the count is flat for 5–10 minutes at an elevated level, you have a leak. If it decays, you do not.
2. Expose goroutine profiles. Add import _ "net/http/pprof" to a binary that already serves HTTP, or write snapshots with runtime/pprof.Lookup("goroutine"). Restrict the endpoint to localhost, an admin port, or an authenticated internal network — pprof exposes sensitive internal state.
3. Take two profiles minutes apart and diff them. From a machine that can reach the pprof endpoint:

   go tool pprof -base http://host:6060/debug/pprof/goroutine?debug=1 \
     http://host:6060/debug/pprof/goroutine?debug=1

   Using -base subtracts the first snapshot, so stacks whose count only increased float to the top. Run this as any user with network access to the debug endpoint; no special permissions on the server are needed.
4. Read the top stacks. Sort by count and look for stacks parked in chan receive/send, select, or IO wait that share an identical call path. The top frames usually point directly at the leaking call site in your code.
5. Apply the fix for that stack (see below), redeploy to staging, and repeat the two-profile diff to verify the count is now stable.

Fixes tied to findings

Blocked channel operations

A goroutine sending on a channel nobody reads parks forever — you cannot kill it from outside, so the blocking operation itself must be made to return. Close producer channels so range loops terminate, and give consumers a cancellation path:

func worker(ctx context.Context, jobs <-chan Job) {
    for {
        select {
        case <-ctx.Done():
            return
        case j, ok := <-jobs:
            if !ok {
                return // producer closed the channel
            }
            process(j)
        }
    }
}

Missing context propagation

If a leaked stack sits inside a database call, HTTP request, or queue consumer, check whether the caller's context.Context actually reaches it. Passing context.Background() at an internal boundary silently severs cancellation and is a frequent leak source.

Unclosed response bodies

Every http.Client response body must be closed, or the underlying connection — and the goroutines reading it — linger:

resp, err := client.Do(req)
if err != nil {
    return err
}
defer resp.Body.Close()

Timers in loops

time.After in a hot loop allocates a timer that lives until it fires, causing transient goroutine and timer buildup under load. Prefer time.NewTimer with an explicit Stop(), or context deadlines. The same applies to tickers: always call ticker.Stop().

Escalation criteria

If the count still grows after fixing the obvious blocked stacks:

• Capture a full dump with /debug/pprof/goroutine?debug=2, which includes goroutine creation stacks — useful when the parking site is generic (e.g., inside a library) but the creator is your code.
• Audit third-party libraries for background goroutines started at init or per-connection; some require explicit Close() to stop them.
• Run a canary with runtime/trace on a short window to watch goroutine lifecycle events before considering deeper runtime investigation.

Verifying the fix

Reproduce locally first: a ten-line program that spawns a goroutine blocked on a channel receive will show runtime.NumGoroutine() climbing and the blocked stack in the profile. Then add a regression test that asserts the count returns to baseline after canceling contexts and closing resources, and run the suite with go test -race. Finally, deploy to staging with go_goroutines scraped and confirm the metric plateaus after traffic stops before promoting. Note that profile labels and frame inlining vary slightly between Go toolchains, so compare stacks by call path rather than exact text.