GROQ Pagination: Transitioning from Unbounded Queries to Sliced Result Sets

Managing Large Datasets in Sanity Content Lake

When scaling a Sanity.io project, retrieving large document collections via the GROQ API requires a shift from unbounded queries to a bounded pagination strategy. The API enforces limits on the number of documents returned per request to maintain performance and prevent timeouts.

Implementing pagination typically involves the slice operator [start...end] to request specific windows of data. However, because this mechanism is stateless, the client must manage the offset indices and potentially execute a separate count() query to determine the total available records for the UI.

As the offset increases during deep pagination, there is a known risk of increased latency since the engine must scan preceding records before returning the requested slice.

• How does the performance of the slice operator scale as the offset index reaches very high values in large datasets?
• What is the recommended strategy to mitigate latency during deep pagination without relying on client-side caching?