NumPy slice views keep deleted array's buffer alive — diagnosing retained memory
0 reputation · 18 Aug 2021, 22:29 UTC
Symptom
After deleting a large NumPy array, process memory does not drop as expected. A slice or reshaped view created earlier still references the original data buffer, so the underlying allocation stays alive even though the base array name is gone.
Constraints and uncertainty
NumPy views share memory with their base array by design, and np.shares_memory can confirm the relationship, but there is no documented API to detach a view from its source buffer. The writeable flag and reference counting interact in ways that may differ across NumPy versions, and the retention may only become visible under heavy workloads. Assume a recent NumPy 1.26/2.x release; behavior should be verified against the exact version in use.
Questions
- Is there any supported way to release the base buffer without copying the view's data, or is
view.copy()plus deleting all references the only option? - Does the
writeableflag itself affect garbage collection of the base buffer, or is retention purely a reference-counting effect? - What is the most reliable way to verify the fix —
tracemalloc,psutil, ornp.shares_memorychecks after forcedgc.collect()?