NumPy ↔ Production Python Runtime: np.load allow_pickle interoperability

Direct answer

The documented default changed at NumPy 1.16. Before 1.16, np.load() used allow_pickle=True. In 1.16 and later, the default is allow_pickle=False. Object-dtype arrays stored in .npy or .npz require pickle to deserialize, so a default load on 1.16 and later typically raises ValueError with a message such as object arrays cannot be loaded when allow_pickle is disabled.

Deserialization becomes policy-dependent rather than format-dependent once the runtime decides whether pickle may run at all. The file format can say dtype=object; the environment decides whether that object path is permitted. If production wraps np.load, blocks pickle use, or forces the flag false, then even explicit allow_pickle=True is not a portability guarantee.

Likely explanation vs confirmed behavior

• Confirmed: NumPy 1.16 changed the default, and object arrays require pickle.
• Likely: Local success plus production failure usually means a version split or a hardened runtime policy, not a corrupt file.

Pickle-free persistence options

1. Normalize object elements into numeric arrays, fixed-width Unicode or byte fields, or structured arrays.
2. Save components in one .npz with np.savez or np.savez_compressed; numeric and string arrays load with allow_pickle=False.
3. For larger or columnar data, use ecosystem formats such as HDF5, Zarr, CSV, or JSON for basic types. NumPy does not make arbitrary Python objects portable in these formats.
4. If an object-array API must remain, reconstruct it with np.empty(shape, dtype=object) and assign validated values after loading the pickle-free parts.

If elements are custom class instances, convert them to basic types or define explicit validated serialization. No NumPy-native pickle-free format preserves arbitrary Python objects automatically.

Minimal verification

import numpy as np
print(np.__version__)
arr = np.array([{'a': 1}, [2, 3]], dtype=object)
np.save('object.npy', arr)
try:
    np.load('object.npy')
except ValueError as exc:
    print('default load blocked:', exc)

Then verify a pickle-free path:

np.savez('parts.npz', ids=np.array([1, 2]), values=np.array([3, 4]))
data = np.load('parts.npz', allow_pickle=False)
print(data['ids'], data['values'])

Decision rule

Use pickle-free storage as the portable default. Reserve allow_pickle=True for a trusted, controlled boundary where runtime policy explicitly permits pickle and data provenance is trusted. One detail changes the recommendation: whether the object elements can be normalized to numbers, strings, bytes, or structured records, or whether arbitrary Python and custom-class instances must survive unchanged.