Diagnosing Neo4j Transaction Log Growth Leading to Disk Full on a Standalone Instance

Recognizable Condition

Neo4j repeatedly logs warnings like WARN: Transaction log file size exceeded threshold. The file /var/lib/neo4j/data/databases/graph.db/transaction.log grows beyond the size set by dbms.tx_log.rotation.size. Monitoring shows the disk partition holding the Neo4j data directory nearing 100 % utilization, and application clients see increased latency or timeouts because Neo4j blocks new transactions while waiting for log space.

Common Causes

Ordered Diagnostic Checks

1. Check log warnings – On the Neo4j host, run:

   grep 'Transaction log file size exceeded threshold' /var/lib/neo4j/log/neo4j.debug.log

   Record timestamps and note the size of transaction.log with:

   ls -lh /var/lib/neo4j/data/databases/graph.db/transaction.log

   (requires read permission on the neo4j logs directory).
2. Verify rotation settings – Inside Neo4j Browser or via cypher-shell execute:

   CALL dbms.listConfig() YIELD name, value WHERE name IN ('dbms.tx_log.rotation.retention_policy','dbms.tx_log.rotation.size') RETURN name, value;

   Note the current values; default retention policy is size and default size is 100M.
3. Identify long‑running transactions – Run:

   CALL dbms.listTransactions() YIELD transactionId, startTime, status WHERE status = 'RUNNING' RETURN transactionId, startTime;

   Any transaction running for minutes or hours is a suspect.
4. Check page cache usage – Look at:

   CALL dbms.query('CALL dbms.component() YIELD name, versions, edition RETURN name')

   and compare dbms.memory.pagecache.size in neo4j.conf with the server’s RAM; a rule of thumb is to allocate 50‑70 % of RAM to page cache.
5. Inspect backup locks – If you run scheduled backups, verify they are not active during the growth period by checking backup logs or running:

   ps -ef | grep neo4j-admin

   while the warning appears.

Fixes Tied to Findings

dbms.tx_log.rotation.retention_policy=size
 dbms.tx_log.rotation.size=100MSave the file and restart Neo4j: 
sudo systemctl restart neo4j
 (requires sudo or neo4j user).

Escalation Criteria

• Disk usage remains above 90 % after applying the above fixes and restarting Neo4j.
• Log file continues to grow despite correct rotation settings and no long‑running transactions.
• Neo4j fails to start after a configuration change, indicating possible store corruption.
• Application experiences persistent timeouts (>30 s) even after log space is freed.

If any of these occur, collect:

• A full copy of /var/lib/neo4j/log/neo4j.debug.log and neo4j.log.
• The output of neo4j-admin info.
• A recent backup of the graph store (if available).
• Then open a support ticket with Neo4j or consult the Neo4j Operations Manual for store recovery procedures.

Verification After Fix

After restarting Neo4j with the corrected settings:

1. Confirm the rotation settings are active:

   CALL dbms.listConfig() YIELD name, value WHERE name = 'dbms.tx_log.rotation.size' RETURN name, value;

2. Monitor the transaction log size for a few cycles (e.g., every 5 minutes) with:

   watch -n 300 ls -lh /var/lib/neo4j/data/databases/graph.db/transaction.log

   You should see the file size reset to roughly the configured dbms.tx_log.rotation.size after each rotation.
3. Check that no new WARN: Transaction log file size exceeded threshold lines appear in neo4j.debug.log.

Limitations and Practical Check

Changing rotation parameters requires a Neo4j restart; during the restart the database is unavailable. Plan the restart within a maintenance window or use a clustered setup for zero‑downtime upgrades. The verification step above assumes you have shell access to the Neo4j host and permission to run ls and Neo4j cypher commands. If you cannot restart immediately, you can temporarily mitigate growth by killing long‑running transactions (via CALL dbms.killTransaction($id) YIELD success RETURN success) but this may roll back work and should be used only as a short‑term measure.