Stop Sending Single Writes: Batching Neo4j Ingests with UNWIND

The Cost of the "One-by-One" Pattern

A common bottleneck in Neo4j applications is the "loop-and-write" anti-pattern. This happens when application code iterates through a dataset and executes a separate MERGE or CREATE statement for every single row. While this feels intuitive, it forces the database to handle a network round-trip, query parsing, and transaction overhead for every individual record.

As your dataset grows from hundreds to thousands of records, throughput collapses. The database spends more time managing the overhead of thousands of tiny transactions than actually writing data to the disk. The solution is to shift the iteration from the application layer to the database engine using UNWIND.

How UNWIND Transforms Ingests

The UNWIND clause takes a list (passed as a single parameter) and "unrolls" it into individual rows. This allows you to send a single request containing a batch of data, which Neo4j then processes as a single unit of work.

Instead of sending 1,000 separate queries, you send one query and one list of maps. This reduces network latency and allows the query planner to optimize the execution of the batch.

The Role of Constraints

Batching alone isn't a silver bullet. If you use MERGE inside an UNWIND block without a uniqueness constraint on the lookup property, Neo4j may perform a full label scan for every row in your batch. To ensure the batch remains fast, you must first create a constraint on the property you are matching against.

Worked Example: Batching Person Records

Assume we are loading a list of users. We are using Neo4j 5.x and an official driver (such as Python or Java). First, ensure the database has a uniqueness constraint to prevent duplicate nodes and enable index seeks:

// Run this once in the Neo4j Browser or via a setup script
CREATE CONSTRAINT person_id_unique IF NOT EXISTS 
FOR (p:Person) REQUIRE p.id IS UNIQUE;

Now, compare these two approaches in your application logic:

The Slow Way (Anti-pattern)

# Pseudocode: Executing one statement per row
for person in person_list:
    session.run("MERGE (p:Person {id: $id}) SET p.name = $name", 
                id=person['id'], name=person['name'])

The Fast Way (UNWIND Pattern)

Pass the entire list as a parameter named $rows. Run this as a single executeWrite transaction:

// Cypher statement
UNWIND $rows AS row
MERGE (p:Person {id: row.id})
SET p.name = row.name

Execution Details:

• Where to run: Application code via the official Neo4j Driver.
• Permissions: The user must have WRITE permissions on the Person label.
• Placeholders: $rows should be a list of maps (e.g., [{id: 1, name: 'Alice'}, {id: 2, name: 'Bob'}]).
• Risk: If one record in the batch causes a constraint violation or error, the entire batch transaction will roll back.

Managing Transaction Size and Memory

While UNWIND is significantly faster, you cannot simply pass a million rows in one list. Every transaction in Neo4j holds a state in memory; an oversized transaction can lead to OutOfMemoryError or cause excessive locking that blocks other operations.

The practical engineering decision is to chunk your data. Depending on your heap size and the complexity of the query, batches of 1,000 to 10,000 records are typically the sweet spot. If you are performing a massive migration and cannot chunk in the application, Neo4j 4.4+ supports CALL { ... } IN TRANSACTIONS, which allows the database to commit sub-batches automatically.

Comparison Summary

Metric	Single Writes	UNWIND Batching
Network Overhead	High (N round-trips)	Low (1 round-trip)
Transaction Cost	High (N commits)	Low (1 commit per batch)
Memory Pressure	Low	Moderate (depends on batch size)
Failure Impact	Single row fails	Entire batch rolls back

Verifying the Result

To verify that your batching is working efficiently, use the PROFILE prefix in the Neo4j Browser with a sample batch:

PROFILE UNWIND [{id: 1, name: 'Test'}] AS row
MERGE (p:Person {id: row.id})
SET p.name = row.name

Look for NodeUniqueIndexSeek in the execution plan. If you see NodeByLabelScan, your constraint is missing, and the UNWIND pattern will slow down as the database grows. To roll back the constraint if it was created in error, use DROP CONSTRAINT person_id_unique.