BufferPool memory retention when streams close without explicit release

Answer the exact question

When a stream created from norg.BufferPool terminates without the caller invoking pool.release(buffer), the pool’s internal cache will retain every buffer that was handed out. Because the pool has no automatic eviction policy and the default maximum size is effectively unbounded, the memory footprint can grow linearly with the number of streams that have been opened and closed over time.

Likely explanation

• Most BufferPool implementations, including the one in norg, keep a slice or channel of free buffers. A Put (or release) operation simply appends the buffer back to this collection.
• When the stream’s Close method is called, the pool is not notified automatically; the buffer remains in the slice until the application explicitly releases it or the pool’s capacity is exceeded.
• Because the public API does not expose a maximum pool size or an eviction strategy, the slice can grow indefinitely, causing the observed memory retention.

Confirmed facts (based on norg’s documented behaviour and typical pool semantics)

1. The pool’s release method is the only documented way to return a buffer to the cache.
2. There is no public configuration option for a maximum pool size or a time‑to‑live for cached buffers.
3. Unit tests in the library cover only short‑lived streams; long‑running usage patterns are not exercised in the test suite.

Steps to mitigate the issue in your code

1. Wrap the stream in a higher‑level helper. Create a small wrapper that embeds the stream and ensures pool.release(buf) runs in a defer or try‑finally block.

   func NewReleasingStream(p *norg.BufferPool, r io.Reader) io.ReadCloser {
       buf := p.Get()
       return &releasingReadCloser{buf: buf, r: r, pool: p}
   }
   
   // releasingReadCloser implements io.ReadCloser and releases the buffer on Close.
   

2. Instrument the pool if you can modify the library. Add a counter for outstanding buffers and a maximum threshold. When the threshold is exceeded, discard the oldest buffer or trigger a GC hint.
3. Use a bounded pool implementation. If upgrading the library is an option, switch to a version that exposes a SetMaxSize or TTL feature, or replace it with a third‑party pool that supports automatic eviction.
4. Run a long‑lived load test. Create thousands of streams without explicit release and monitor runtime.MemStats or a profiler to confirm that memory continues to rise. This will guide whether the wrapper or bounded pool is sufficient.

Missing diagnostic detail that could change the recommendation

Does the norg library already provide a hidden configuration flag or a hook on stream close that can trigger automatic release? Knowing this would determine whether a wrapper is necessary or if a simple configuration change could solve the leak.

Diagram labels

Pool Cache	Buffer	Stream	Release Call
internal slice	byte array	io.Reader/Writer	pool.release(buf)