NumPy memmap: Undefined behavior when underlying file size changes

Direct answers

1. No. np.memmap does not raise ValueError when the backing file is truncated after mapping. The mapping length is fixed at construction, and NumPy does not poll the file. Accessing a truncated region is operating-system dependent and may return stale pages, fail with an access error, or terminate the process with a signal such as SIGBUS on POSIX, often without a Python traceback. Do not design around a future automatic ValueError; implement size tracking outside NumPy if needed.
2. No automatic refresh exists. Appending bytes to the backing file does not change the existing mapping. Close the current np.memmap and reopen it with the new size or offset to see appended data; that is the reliable NumPy-level approach.
3. No. NumPy provides no built-in locks, atomicity, or consistency guarantees for multiple processes writing to the same memmap. Use external synchronization, and coordinate any file-size change outside NumPy.

Likely explanation versus baseline behavior

np.memmap builds on the operating system's mapped-file facility and stores length, shape, dtype, and offset when created. It does not re-stat the file on access, so mode only changes access permissions and cannot add truncation detection or automatic refresh. Treat fixed mapping length, invisible extension, undefined or OS-faulting truncation, and torn interleaved writes as the baseline; exact fault behavior differs between POSIX and Windows.

Steps for this case

1. Do not rely on an open np.memmap after another process may change the file size.
2. Close and reopen the memmap when you need to observe growth or after a controlled truncation; recreate the array shape if it depends on file size.
3. Use an external lock around file-size changes and memmap access. On POSIX, fcntl.flock or fcntl.lockf is a common choice; use the equivalent file-locking API on Windows.
4. Serialize writes to overlapping regions. Coordinate truncation or extension even for disjoint regions because remapping can invalidate other processes' mappings.

Verification

Run this in a throwaway script, not a long-running service, because truncating a mapped file can crash the process:

import os, tempfile, numpy as np
fd, path = tempfile.mkstemp(suffix='.bin')
os.close(fd)
with open(path, 'wb') as f:
    f.write(b'\x00' * 1024)
mm = np.memmap(path, dtype=np.uint8, mode='r', shape=(1024,))
# Scratch only: do not truncate a mapped file in a service.
# os.truncate(path, 512)
# print(mm[600])
mm.close()
with open(path, 'ab') as f:
    f.write(b'\x01' * 64)
mm = np.memmap(path, dtype=np.uint8, mode='r', shape=(os.path.getsize(path),))
print(int(mm[-1]))
os.remove(path)

The commented truncation lines illustrate the unsafe case. The reopen block verifies that appended bytes appear only after a new np.memmap is created.

Assumptions and uncertainty

• Assumes a standard NumPy build using POSIX mmap or Windows file mapping; behavior is platform-dependent.
• No specific NumPy version is assumed. Exact signals or exceptions for truncated mappings are not guaranteed by NumPy and may vary by OS and filesystem.
• OS, NumPy version, and a minimal repro can help interpret a live crash, but they do not change the mitigation above.