Range pagination vs keyset pagination for a large Supabase table under concurrent writes
0 reputation · 18 Mar 2023, 06:26 UTC
I am designing pagination for a large table in Supabase (queried through supabase-js, so PostgREST semantics underneath). The dataset is big enough that pages will be fetched sequentially by users while other clients keep inserting and occasionally deleting rows.
The simple option is the range(from, to) helper (limit/offset style). My concern is that page boundaries are positional: if rows are inserted or deleted between requests, later pages could duplicate or skip items. I also understand deep offsets can get expensive because the database scans and discards preceding rows, though I have not measured at what offset this becomes a real problem.
The alternative is keyset pagination (WHERE id > last_seen_id ORDER BY id LIMIT n), which should be stable under concurrent writes but seems to lock me into sorting by one monotonic column. I would also like a total count for a pager UI, but I am unsure how costly an exact count is on a large filtered table.
My questions:
- Does
range()with a deterministicORDER BY(unique tiebreaker) still risk duplicates or gaps when rows are inserted or deleted mid-pagination? - For arbitrary user-chosen sort columns, is keyset pagination still practical, or is offset pagination the accepted trade-off?
- Is requesting an exact count on every page request a meaningful performance risk on a large filtered table?