Racket/db LIMIT/OFFSET pagination: performance uncertainty with large offsets
0 reputation · 22 Jun 2024, 12:22 UTC
Goal
Assess whether using large OFFSET values in query calls from the racket/db library causes measurable performance regressions across common back‑ends (SQLite, PostgreSQL, MySQL). The library passes raw SQL strings with LIMIT and OFFSET unchanged, but the effect of high offset values on driver and engine execution plans is unclear.
Constraints & Uncertainty
1. The dataset size is expected to be in the hundreds of thousands. 2. Offset values may exceed 50 000 or more. 3. The same SQL text is sent to different DBMS drivers, each with its own execution strategy for OFFSET. 4. The racket/db library does not provide a built‑in keyset pagination abstraction, so developers must decide between offset‑based paging or manual WHERE‑clause pagination.
Specific Questions
- Does
racket/dbforwardLIMITandOFFSETverbatim to SQLite, PostgreSQL, and MySQL drivers, or are there subtle differences? - What is the observed query latency when
OFFSETexceeds 100 000 rows on each back‑end, and how does it compare to keyset pagination using a unique ordered column? - Is there a recommended pattern or helper in the Racket ecosystem to implement efficient keyset pagination without manual SQL construction?