Neo4j Full-Text Search: When to Use Native Indexes Over External Engines

The Problem: Text Search in Graph Queries

You're building a knowledge graph where users search for "machine learning" across research papers, authors, and citations. A Cypher WHERE paper.abstract CONTAINS 'machine learning' works for hundreds of nodes but crawls at thousands. The query planner shows NodeByLabelScan with dbHits proportional to every node of that label—no index can help a substring predicate on a property.

Neo4j's native full-text indexes (backed by Lucene) turn that O(N) scan into a sub-millisecond lookup. The trade-off: eventual consistency and storage overhead. Here's how to decide if they fit your workload.

How Native Full-Text Indexes Work

Create an index on one or more labels and properties:

CALL db.index.fulltext.createNodeIndex('paperAbstractIdx', ['Paper'], ['abstract', 'title'])

Run this in the Neo4j Browser or cypher-shell with admin or index_manager role. The procedure returns {name, type, entityType, labels, properties} on success. Verify with CALL db.index.fulltext.list().

Each indexed property is tokenized by the standard analyzer (Unicode text segmentation, lowercasing, stop-word removal). You can specify a custom analyzer per property if you need keyword, whitespace, or language-specific tokenization.

Query Pattern and Scoring

Search returns nodes with a relevance score:

CALL db.index.fulltext.queryNodes('paperAbstractIdx', 'machine learning')
YIELD node, score
RETURN node.title, node.year, score
ORDER BY score DESC
LIMIT 20

The score is Lucene's TF-IDF variant—higher means more relevant. You can combine with graph traversals:

CALL db.index.fulltext.queryNodes('paperAbstractIdx', 'transformer architecture')
YIELD node AS paper, score
MATCH (paper)<-[:AUTHORED]-(author:Researcher)
RETURN paper.title, author.name, score
ORDER BY score DESC

This avoids scanning all :Paper nodes; the index yields candidate nodes first, then the traversal expands.

Worked Example: Benchmarking the Difference

Create 50,000 :Paper nodes with abstract (average 300 words). Measure two approaches:

1. Property scan: PROFILE MATCH (p:Paper) WHERE p.abstract CONTAINS 'neural network' RETURN p LIMIT 10
2. Full-text index: PROFILE CALL db.index.fulltext.queryNodes('paperAbstractIdx', 'neural network') YIELD node RETURN node LIMIT 10

Expected outcome (observed on Neo4j 5.x, 8 GB heap, SSD):

Metric	CONTAINS Scan	Full-Text Index
dbHits	~50,000	~200
Elapsed time (cold)	1,200 ms	15 ms
Elapsed time (warm)	800 ms	2 ms

The index reduces dbHits by two orders of magnitude. The PROFILE output shows NodeIndexSeek instead of NodeByLabelScan.

Trade-offs and Limitations

• Eventual consistency: A node created in transaction tx1 may not appear in a full-text query until tx1 commits and the index refreshes (typically < 100 ms). For strict read-your-writes, either wait or re-query after commit.
• Storage growth: Each unique token adds to the Lucene segment files. In the 50k-paper test, the index grew to ~1.2 GB vs. 300 MB for the base store. Monitor with CALL dbms.listConfig() YIELD name, value WHERE name CONTAINS 'fulltext' RETURN name, value and filesystem usage.
• Write throughput impact: Heavy ingest (10k+ nodes/sec) can cause index writer contention. Batch commits and consider db.index.fulltext.awaitIndexing('paperAbstractIdx', 30, TimeUnit.SECONDS) after bulk loads.
• No transactional rollback: Dropping an index (CALL db.index.fulltext.drop('paperAbstractIdx')) is immediate and irreversible. Back up the database first if the index is large.

When to Choose Native vs. External Search

Use Neo4j's native full-text index when:

• Text search is a filter before graph traversal (the example above).
• You need ACID guarantees on the graph side and can tolerate < 1 s index lag.
• Operational simplicity matters—no separate Elasticsearch/OpenSearch cluster to manage.

Offload to a dedicated search engine when:

• You need advanced features: fuzzy matching, synonym expansion, per-field boosting, or real-time analytics dashboards.
• Text fields exceed 100 KB per property (Lucene's default term byte-length limit).
• Write volume demands horizontal scaling of the search tier independent of the graph.

Verification Checklist

1. Create the index and confirm it appears in db.index.fulltext.list().
2. Insert test nodes with known terms, run a query, and verify score > 0 for matches.
3. Run PROFILE on your production query pattern before and after index creation; confirm dbHits drops and plan shows NodeIndexSeek.
4. Monitor index size weekly for the first month; set an alert if growth exceeds 2× the source property size.

If the index meets latency and storage targets, keep it native. If you hit the limitations above, plan a migration to an external search cluster—Neo4j's apoc.export.json or CDC connectors can stream changes.