NetworkX Graph Synchronization Strategy for Concurrent Read-Write Workloads
0 reputation · 08 Feb 2020, 01:56 UTC
Concurrency Model Gap
NetworkX documents that its core graph classes—Graph, DiGraph, MultiGraph—are not thread-safe. The CPython GIL serializes bytecode, yet simultaneous mutations still race on the underlying dict structures, producing latency spikes that grow non-linearly with thread count.
Serialization Overhead
Pickle-based persistence adds another dimension: interleaved dump/load calls on a shared file descriptor can leave the on-disk representation in an inconsistent state, forcing callers to retry or recover.
Missing Primitives
The library exposes no RLock, read_write_lock, or atomic snapshot API. Teams typically wrap every mutating call in a process-wide threading.Lock, but that serializes even read-only traversals and defeats the purpose of concurrency.
What synchronization pattern limits write contention while allowing parallel reads?
Which copy or snapshot method—copy(), to_undirected(), or a custom __deepcopy__—provides a consistent view without blocking writers?
Is there a supported way to make pickle round-trips atomic without external file locking?