Question
OpenCV C++ core and Python cv2 binding: Mat ownership and deallocation across the language boundary
Neel Moss
0 reputation · 14 Dec 2023, 04:26 UTC
149.4K views
Loading…
0 reputation · 14 Dec 2023, 04:26 UTC
The integration goal is to define clear ownership and release timing for cv::Mat objects that cross from OpenCV C++ core into the Python cv2 binding.
The C++ core uses reference-counted cv::Mat with shallow copy semantics. The Python binding wraps a Mat in a capsule with a custom deallocator that decrements the C++ refcount when the Python object is finalized. Ownership is transferred on the C++ to Python boundary for functions returning Mat, and the Python wrapper holds the sole reference unless additional references are kept. UMat and GPU-backed Mats add a device memory layer whose lifetime is tied to both the C++ refcount and the Python wrapper lifetime. Behavior is version-sensitive between OpenCV 3.x and 4.x binding generation and varies with build options for OpenCL and CUDA.
OpenCV does not define a deterministic release contract for Mats created in C++ extensions and used from Python; explicit release is documented as available but not required. This leaves expectations about immediate native free versus GC-deferred free ambiguous across bindings.
What is the documented ownership transfer rule for Mats returned from C++ to Python and when is native memory guaranteed to be freed? Is explicit release required for deterministic deallocation of Mats originating in C++ extensions? How does UMat device buffer lifetime relate to Python wrapper finalization and C++ refcount?
35275 reputation · 14 Dec 2023, 12:00 UTC
When a cv::Mat is returned from a C++ extension to Python via the cv2 bindings, ownership is managed through a shared reference-counting mechanism. The C++ cv::Mat is wrapped into a NumPy array. The Python object does not copy the image data; instead, it maintains a pointer to the original C++ data buffer and increments the internal C++ reference count.
Native memory is guaranteed to be freed only when the C++ reference count reaches zero. In a standard return scenario, this occurs when the Python NumPy object is finalized by the Python Garbage Collector (GC). Because Python's GC is non-deterministic, the actual release of native memory may be deferred until a collection cycle is triggered, even if the Python variable is deleted or goes out of scope.
Explicit release is not required for memory safety, but it is necessary for deterministic deallocation. Because the Python wrapper holds a reference to the C++ cv::Mat, calling del on the Python object only decrements the Python-side reference count. It does not force the immediate destruction of the C++ object if other references exist (e.g., views created via NumPy slicing).
To encourage faster release in memory-constrained environments, you can:
opencvdel image_var to remove the Python reference.gc.collect() to trigger the finalization of the wrapper and the subsequent decrement of the C++ refcount.The lifetime of cv::UMat (Universal Mat) buffers follows the same reference-counting logic as cv::Mat, but with an added layer of abstraction for OpenCL/GPU memory. The device buffer remains allocated as long as at least one cv::UMat or cv::Mat header points to it.
When the Python wrapper for a UMat is finalized, it decrements the C++ reference count. If this is the final reference, the C++ destructor is called, which signals the OpenCL runtime to release the device memory. However, if the UMat was converted to a Mat (CPU access) within the C++ core before being passed to Python, the memory may stay pinned in a state that prevents immediate device-side freeing until all CPU-side views are also destroyed.
To verify that a Python object is sharing memory with the C++ core rather than holding a copy, compare the data pointers:
import cv2
# Assuming 'img' is returned from your C++ extension
print(img.__array_interface__['data']) # Check the memory address in PythonIf the address matches the mat.data pointer in your C++ logs, the shared ownership model is active.
Use comments to ask for clarification. Post a solution as an answer.
35,275 reputation · 14 Dec 2023, 15:55 UTC
To expand on the deterministic deallocation point, it is important to note how NumPy's view mechanism interacts with the underlying cv::Mat reference count. When you create a slice of a Python image (e.g., roi = image[y:y+h, x:x+w]), NumPy creates a new array object that shares the same data buffer as the original.
In the cv2 binding, this means the internal C++ reference count is not decremented even if the original image variable is deleted. The native memory remains allocated as long as any NumPy view—no matter how small the slice—exists. To verify this behavior in a memory-constrained environment, you can inspect the .base attribute of the sliced array; if it points back to the original array, the C++ buffer is still pinned.
For those using OpenCV 4.x, ensuring that all views are cleared before calling gc.collect() is the only way to guarantee the C++ refcount can actually reach zero.