Unexpected heap growth during sustained write transactions despite stable page cache usage

The steady increase in on-heap memory during sustained writes is typically caused by the accumulation of **Transaction State**, not a failure of the page cache eviction policy. When writes occur, Neo4j must keep all modified nodes, relationships, and properties in the heap to ensure ACID compliance and isolation until the transaction is committed.

The Interaction Between Heap and Page Cache

Neo4j uses a Least Recently Used (LRU) eviction policy for the page cache. While the page cache manages data pages read from disk (off-heap), the specific changes made by an active transaction are tracked on-heap.

• State Retention: Every change in a long-running transaction creates objects on the heap. If the transaction remains open-open, these objects cannot be garbage collected, leading to heap growth.
• Dirty Page Eviction: Dirty pages in the page cache are not necessarily evicted later than clean pages; however, the system cannot safely flush them to the data store until the transaction is committed or a checkpoint occurs.
• Memory Pressure: If the page cache is stable, it has reached its fixed limit. The heap grows because it is busy holding the "delta" of your writes that have not yet been finalized.

Steps to Resolve and Verify

To address heap growth in heavy write scenarios, follow these steps:

1. Batch Transactions: Refactor large write operations to use smaller, atomic batches. This allows the garbage collector to reclaim memory used by transaction state after each batch commits.
2. Monitor Transaction State: Use the JMX metric neo4j.memory.transaction.state.used. If this metric scales linearly with your heap, the issue is the transaction duration or size.
3. Adjust Heap Allocation: Ensure dbms.memory.heap.max_size is sufficient to hold the peak expected uncommitted transaction state plus the overhead for the query engine.
4. Check Checkpoint Frequency: Review the dbms.checkpoint.interval_interval. While checkpoints flush dirty pages to disk, they do not clear the on-heap transaction state of an active session.

To provide a more specific recommendation, could you clarify if the heap growth occurs within a single long-running transaction or across thousands of small concurrent transactions?