Tuning refresh interval vs. search after for high-concurrency latency

Refresh Intervals and Cursor Stability

Increasing the index.refresh_interval does not destabilize search_after cursors. Because search_after relies on the sort values of documents already committed to a searchable Lucene segment, the cursor remains valid regardless of how often new segments are created. The primary impact is visibility lag: documents indexed after a refresh cycle will not appear in any pagination sequence until the next refresh occurs.

The Relationship Between Refresh and Latency

Under high-concurrency write loads, frequent refreshes force the creation of many small segments. This triggers aggressive background merging, which consumes significant CPU and I/O, often leading to thread pool exhaustion and P99 latency spikes. By increasing the refresh interval, you reduce the merge frequency, thereby stabilizing the system's resource availability for search queries.

Implementation Steps for High-Concurrency Tuning

1. Increase Refresh Interval: Move from the default 1s to a higher value (e.g., 30s or 60s) to reduce segment churn.

   PUT /my_index/_settings
   {
     "index": {
       "refresh_interval": "30s"
     }
   }

2. Implement Point-in-Time (PIT): If your application cannot tolerate documents shifting positions or disappearing between pagination requests due to updates/deletes, pair search_after with a PIT. This ensures the cursor views a consistent snapshot of the index.
3. Monitor Thread Pools: Use the _nodes/stats API to verify that the write and search thread pool queues are no longer backing up.

Performance Thresholds

There is no single documented "magic number" for the refresh interval, as the optimal threshold depends on your hardware's I/O throughput and document size. However, the gain in merge performance generally outweighs the cost of staleness when the indexing rate is high enough to cause constant segment merging. If your P99 latency is driven by merge I/O wait, any interval that reduces the number of segments created per minute will provide a measurable gain.

Missing Diagnostic Detail: Are you currently using a Point-in-Time (PIT) ID with your search_after requests? If not, your instability may be caused by document updates/deletes rather than the refresh interval.