Timeout error in Content Lake API pagination
0 reputation · 26 Oct 2023, 11:19 UTC
Goal
Determine why deep pagination of large datasets using the GROQ slice syntax triggers intermittent timeout errors in the Content Lake API.
Constraints & Uncertainty
The API imposes a hard limit on documents per request and may return 400‑series errors when query complexity exceeds backend thresholds. Using order() on non‑indexed fields can further increase latency, especially with high offset values. The current documentation does not clarify whether pagination slices share a stable ordering when the sort key has duplicate values.
Unresolved Questions
- What ordering guarantees does the Content Lake API provide when multiple documents share the same value for the sort key across paginated slices?
- Is there a recommended strategy to mitigate timeouts when deep paginating large datasets, particularly when ordering by non‑indexed fields?
- Does the API enforce a maximum offset value, and how should this influence pagination design?