Question
Confluence Data Center and Apache Lucene: Asynchronous Indexing Latency
Zo Cedar
0 reputation · 27 Dec 2024, 12:22 UTC
views
Loading…
0 reputation · 27 Dec 2024, 12:22 UTC
Confluence Data Center relies on an inverted index powered by Apache Lucene to facilitate full-text search across pages and attachments. To maintain application responsiveness, the indexing process is handled asynchronously via a background job queue, meaning content updates are not immediately reflected in search results.
In environments with high content volume or large attachments, the gap between a page update and its availability in the Lucene index can vary. This creates a challenge for workflows that require immediate verification of content visibility across distributed shards.
What is the expected behavior for index synchronization across shards during peak CPU utilization? Is there a documented mechanism to prioritize specific content updates within the asynchronous queue to reduce retrieval latency?
34775 reputation · 27 Dec 2024, 14:10 UTC
In Confluence Data Center, the Lucene index is not sharded across nodes. Instead, each node maintains a local Lucene index that synchronizes with a shared store (database or file system). During peak CPU utilization, background indexing threads compete for resources with application threads, which can increase the latency between content updates and their appearance in search results.
The asynchronous indexing queue processes updates in the order received. There is no documented mechanism to prioritize specific content updates—all jobs follow FIFO processing. Under high CPU load, indexing delays compound because the queue cannot drain as quickly as new updates arrive.
Confluence exposes index refresh intervals in Administration > System > Indexing. Default values range from 1–5 minutes depending on version. Reducing this interval decreases latency but increases CPU usage during normal operation.
To force immediate synchronization, use the manual Reindex operation from the same administration page. This bypasses the queue and rebuilds the index synchronously, but it is resource-intensive and should only be used when necessary.
confluence/rest/api/indexing/status to view queue depth and last processed timestamp.This guidance assumes you are running Confluence Data Center 7.13 or later. If you are using an older version, indexing behavior may differ. Please confirm your Confluence version so recommendations can be scoped appropriately.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.