V8 Heap Limit Configuration for Large Dataset Pagination

Memory Management for High-Volume Data Processing

When processing large datasets through paginated queries in a V8 environment (such as Node.js), the engine's generational garbage collection strategy manages objects across the New Space and Old Space. While short-lived objects are handled by the Scavenger, objects that persist across pagination cycles are promoted to the Old Space.

A challenge arises when configuring --max-old-space-size to balance application throughput against the latency introduced by Major GC (Mark-Sweep-Compact) cycles. If the heap limit is set too high, the engine may delay full garbage collection, potentially leading to increased pause times that impact responsiveness during heavy data ingestion.

Given these memory dynamics, what are the optimal constraints for bounding the Old Space to ensure incremental marking effectively mitigates main-thread pauses during large-scale pagination? How does the promotion rate from New Space to Old Space affect the frequency of Full GC events under these specific configurations?