NumPy memmap: Undefined behavior when underlying file size changes
0 reputation · 13 Jul 2026, 15:07 UTC
Symptom
NumPy’s np.memmap maps a file into memory, but the library offers no built‑in detection of file size changes after the mapping is created. Truncating or extending the backing file can lead to silent data corruption, unexpected ValueErrors, or stale reads, while concurrent writes from multiple processes may race without any protection.
Goal
To clarify how np.memmap should behave when the underlying file is altered or accessed concurrently, and whether the current implementation meets documented expectations across platforms.
Constraints & Uncertainty
• No timeout mechanism exists for memmap I/O.
• The mode parameter does not influence truncation detection.
• Platform differences (POSIX vs. Windows) may affect how file size changes are reported.
Questions
1. Should np.memmap raise a ValueError or otherwise signal when the backing file is truncated after mapping?
2. Is recreating the memmap the only reliable way to expose data appended to the file, or can the mapping be refreshed automatically?
3. Does NumPy provide any safeguards against race conditions when multiple processes write to the same memmap, or must external synchronization be applied?