Optimizing Text Search in Neo4j: Choosing the Right Full-Text Index Strategy

The Problem: Linear Scans vs. Full-Text Indexes

Searching for specific words or phrases within large string properties using standard Cypher predicates like CONTAINS or STARTS WITH triggers a full scan of all nodes with that label. As your dataset grows to millions of nodes, these queries become prohibitively slow. To achieve sub-second search latency, you must move from property scanning to a full-text index, which tokenizes text into a searchable inverted index.

Decision Matrix: Selecting an Analyzer

The primary decision when implementing full-text search is choosing the analyzer. The analyzer determines how text is broken into tokens (words) and whether those tokens are modified (stemmed) to match variations of a word.

Analyzer	Behavior	Best Use Case	Trade-off
standard	Splits by whitespace/punctuation; no stemming.	Technical IDs, SKU codes, exact keyword matches.	No linguistic intelligence (e.g., "run" won't match "running").
english	Removes stop words; applies English stemming.	Natural language documents, product descriptions.	Incorrectly stems non-English words.
french	Removes stop words; applies French stemming.	French language content.	Limited to French linguistic rules.
custom	User-defined tokenization/stop-word lists.	Medical, legal, or highly specialized domains.	Requires Enterprise Edition and manual config.

Engineering Trade-offs

Storage vs. Speed: Full-text indexes are separate data structures. They increase disk usage and memory pressure proportional to the volume of text indexed. While they accelerate reads, they introduce a slight overhead to write operations as the index must be updated synchronously.

Precision vs. Recall: The standard analyzer provides high precision (you get exactly what you asked for) but low recall. Language-specific analyzers like english increase recall by matching word roots, but may occasionally return irrelevant results due to over-stemming.

Implementation Guide (Neo4j 4.4+)

To implement a full-text index, you must define the index on specific labels and properties. Run these commands in the Neo4j Browser or cypher-shell with administrative permissions.

1. Create the Index

In this example, we index a Product node's name and description using the English analyzer to support natural language queries.

CREATE FULLTEXT INDEX productSearch FOR (p:Product) 
ON EACH p.name, p.description
OPTIONS {analyzer: 'english'};

2. Execute the Search

Full-text indexes are not queried via standard MATCH clauses. Instead, use the db.index.fulltext.queryNodes procedure.

CALL db.index.fulltext.queryNodes('productSearch', 'wireless headphones') 
YIELD node, score
RETURN node.name AS Name, node.description AS Description, score
ORDER BY score DESC;

Note: The score is a Lucene-based relevance value. Higher scores indicate a stronger match based on term frequency and inverse document frequency.

3. Validation and Diagnostics

To verify that the index is operational and not still in the "POPULATING" state, run:

SHOW INDEXES WHERE name = 'productSearch';

Check the state column; it must be ONLINE before queries will return complete results. To check the actual index size and population percentage, use:

CALL db.indexes.list() YIELD name, type, populationPercent, state 
WHERE name = 'productSearch' 
RETURN *;

Limitations and Constraints

• Property Types: Only string and string[] (arrays of strings) are supported.
• Edition Lock: Custom analyzer configurations are restricted to the Enterprise Edition.
• Write Impact: Heavy write bursts to indexed properties can cause temporary latency spikes in transaction commit times.

Rollback Procedure

If the index consumes too much disk space or degrades write performance, remove it using the following command:

DROP INDEX productSearch;