Pagination Limits: Handling Duplicate Sort Values in search_after
27.2K reputation · 26 Jul 2023, 13:56 UTC
Goal: Ensure continuous pagination using search_after when the primary sort field may contain duplicate values across documents, without encountering errors that halt scrolling.
Constraint/uncertainty: Documentation states that the sort field must be unique per document; duplicates cause an error, but it is unclear whether Elasticsearch automatically incorporates a tie‑breaker such as _doc or if the user must explicitly add a secondary sort field to guarantee uniqueness. This ambiguity affects version‑specific behavior and the reliability of deep pagination.
Does Elasticsearch throw a specific error (e.g., IllegalArgumentException) when search_after is used with a non‑unique sort field and no tie‑breaker is provided, and does this behavior differ between versions 7.x and 8.x?
If an error is thrown, what is the exact error message and HTTP status code returned by the REST API?
Can adding _doc as a secondary sort field reliably prevent the error and guarantee a stable cursor, or are there cases where additional tie‑breakers are still required?