Elasticsearch Pagination: When to Use search_after Instead of from+size

Problem and takeaway

You need to page through Elasticsearch results, and queries work on the first few pages but slow down or fail as the offset grows. The useful takeaway: use search_after for sequential deep pagination, such as infinite scroll or log exports, and restrict from + size to shallow pages inside index.max_result_window, which defaults to 10,000. Raising that limit removes the hard rejection but does not remove the memory and merge cost of deep offsets.

Terms used below:

• result window: the range of documents Elasticsearch will consider for a single sorted search, capped by the index-level setting index.max_result_window. The coordinator enforces this limit; each shard computes its own top hits and sends them upward.
• shard: one Lucene index that stores a subset of an Elasticsearch index's data.
• coordinator node: the node that receives the request, forwards it to shards, merges partial results, and returns the final page.
• tie-breaker: an additional sort field that guarantees a deterministic order when primary sort values are equal.
• cursor: the sort values from the last document on a page, passed as search_after to fetch the next page.

This guide assumes Elasticsearch 7.x or 8.x. Defaults and API behavior can differ in older versions, and index.max_result_window can be changed on any cluster, so confirm your cluster's actual setting before relying on the default.

Decision and constraints

Choose the API based on how users move through results:

• Sequential access: start at the beginning and move forward one page at a time, as with infinite scroll or log exports.
• Random access: jump directly to page 1,000 or page 5,000, as with numbered pagination links.
• Deterministic order: every document has a predictable position in the sort, even when two documents share the same timestamp.
• Cluster heap: deep queries consume memory on coordinator nodes and data nodes.

If your interface offers numbered page links deep into the result set, you need random access. If it offers next-page navigation, you have sequential access. That distinction determines which API is safe.

Supported options compared

Option	Best for	Arbitrary page jump	Required sort	Default boundary	Primary risk
from + size	Shallow pages, direct access to page N	Yes	None required; relevance-only sort is unstable across pages	index.max_result_window, default 10,000	Heap pressure and coordinator merge cost grow with offset
search_after	Sequential feeds, exports, infinite scroll	No	Deterministic sort with unique tie-breaker such as _id	None; cursor-based	Skipped or duplicated documents without a unique tie-breaker; cursor values must match sort fields in order and type

Trade-offs

from + size

from + size tells each shard to sort its matching documents and return offset plus size hits. The coordinator merges these lists and discards everything before the requested offset. As offset grows, each shard must track and transport more documents, and the coordinator must sort a larger priority queue. The cost rises with offset and shard count; describing it as roughly proportional to offset times size is a heuristic, not a guaranteed formula, because actual cost depends on shard count, cluster topology, and query complexity. That is acceptable for page 2, but expensive for page 10,000.

search_after

search_after uses the sort values of the last document on the current page as a cursor. Elasticsearch resumes from that point instead of re-scanning earlier documents. It cannot jump directly to page N, and it cannot sort reliably by _score alone because scores are calculated per shard and are not globally unique or stable across pages.

Implementation: search_after for sequential pages

Run this in Kibana Dev Tools or with curl against the _search endpoint. Required permission: read access to the target indices; where security is enabled, the corresponding search privilege, whose exact name varies by version and configuration. Replace logs-2026.10*, the timestamp, and the ID with real values from your data.

GET /logs-2026.10*/_search
{
  "size": 100,
  "sort": [
    { "@timestamp": { "order": "asc" } },
    { "_id": { "order": "asc" } }
  ],
  "search_after": [
    "2026-10-01T12:00:00Z",
    "AbCdEf123456"
  ],
  "query": {
    "range": {
      "@timestamp": {
        "gte": "2026-10-01T00:00:00Z"
      }
    }
  }
}

Expected check: the response contains hits.hits with at most size documents. Take the sort array from the last hit and submit it as search_after in the next request:

"search_after": [
  "2026-10-01T12:05:30Z",
  "FgHiJk789012"
]

Risks: omitting _id as a tie-breaker means documents with identical timestamps can be skipped or repeated across pages. The cursor values must match the sort fields exactly in order and type; for example, if the first sort field is a date, the first cursor element must be that date value. Sort fields should use doc_values where applicable; _id is supported as a tie-breaker without doc_values configuration. Do not sort by _doc across shard relocations or refreshes; the order is not stable.

Implementation: from+size for shallow pages only

GET /logs-2026.10*/_search
{
  "from": 0,
  "size": 100,
  "sort": [
    { "@timestamp": { "order": "asc" } },
    { "_id": { "order": "asc" } }
  ]
}

Use from + size when users must jump to an arbitrary page number and the page stays below the active result window. Do not use it for exports or infinite scroll over millions of documents.

Validation: confirm the active result-window limit

Before testing deep offsets, inspect the setting. Required permission: read access to index settings.

GET /logs-*/_settings?include_defaults=true&pretty

Look for index.max_result_window. The default is 10000, but your cluster may have changed it.

Probe the boundary. The limit applies to from + size, so at the default:

GET /logs-*/_search
{
  "from": 9990,
  "size": 10,
  "sort": [{ "_id": { "order": "asc" } }]
}

Expect HTTP 200 because from + size equals 10,000.

GET /logs-*/_search
{
  "from": 9991,
  "size": 10,
  "sort": [{ "_id": { "order": "asc" } }]
}

Expect HTTP 400 with a result_window error because from + size equals 10,001, unless the limit was raised.

Raising the limit changes cluster state

If your application truly requires random access beyond 10,000 hits, update the dynamic index setting:

PUT /logs-*/_settings
{
  "index": {
    "max_result_window": 50000
  }
}

Required permission: write access to index settings. Rollback: repeat the same request with 10000 to restore the default. No index restart is required in Elasticsearch 7.x and 8.x, but the new value applies to all shards on all nodes. Raising the limit increases heap usage during deep from + size queries and can trigger circuit-breaker errors. Test on staging first.

Validation: check for duplicates and skips with search_after

Run two consecutive search_after requests using the same sort order with _id as the tie-breaker. Collect the _id value from every hit on both pages. The two sets should not overlap, and the second page should begin immediately after the last document of the first page. If you see repeated IDs or missing IDs, the sort order is not deterministic.

Limitations

• search_after does not support jumping to an arbitrary page number.
• search_after should not sort by _score alone; include _id or another unique field.
• search_after is not stable when sorting by _doc across shard relocations or refreshes; use an indexed field plus _id.
• Raising index.max_result_window does not make deep from + size cheap; it only removes the hard rejection.
• Exact security privilege names vary by Elasticsearch version and cluster configuration; confirm the required privileges against your cluster's security setup.

Practical way to check the result

1. Run GET /logs-*/_settings?include_defaults=true and confirm index.max_result_window.
2. Run from: 9990, size: 10 and confirm success, then from: 9991, size: 10 and confirm rejection at the default limit.
3. Run two search_after requests with sort [@timestamp, _id] and verify no repeated _id values across pages.
4. Compare took between shallow from + size and search_after at equivalent offsets to observe the cost difference. Treat took as a heuristic because it is affected by caching and cluster load.