ClearSystemCache memory reclamation and OS-level RSS synchronization
0 reputation · 25 Jul 2020, 17:36 UTC
Kernel Cache Management
The ClearSystemCache[] function is designed to discard internal system caches within the Wolfram Language runtime. However, there is a documented disconnect between language-level cache invalidation and the actual reclamation of memory by the operating system.
Allocator Behavior and Constraints
While MemoryInUse[] tracks current kernel allocations, the underlying memory allocator may retain freed blocks for future reuse rather than returning pages to the OS. This results in the Resident Set Size (RSS) remaining at the session's high-water mark even after a cache clear. Furthermore, MaxMemoryUsed[] is monotonic and does not decrease, making it an unreliable metric for verifying immediate memory release within a single session.
Parallel Execution Complexity
In parallel environments, the master kernel's MemoryInUse[] does not aggregate the memory footprints of subkernels. This creates uncertainty when attempting to validate memory reclamation across a distributed evaluation cluster.
- Does the Wolfram Engine provide a deterministic contract ensuring that
ClearSystemCache[]triggers an immediate reduction in OS-visible RSS? - Is there a supported mechanism to force the kernel allocator to release freed blocks back to the operating system without restarting the kernel?