Which pagination strategy fits a 40M-row Phalcon table: QueryBuilder paginator COUNT cost or manual keyset?
0 reputation · 06 Feb 2022, 11:45 UTC
I am designing the read path for a Phalcon 5 application backed by an InnoDB table expected to reach roughly 40 million rows. The UI is a classic page-number grid today, but an infinite-scroll view is on the roadmap, so I want to pick a pagination approach that will not need to be thrown away later.
My understanding is that Phalcon\Paginator\Adapter\QueryBuilder applies LIMIT/OFFSET per page and issues a separate COUNT query to compute total_pages. On a table this size I expect both the deep-OFFSET scans and the COUNT itself to become the dominant cost, and the total may be stale under write load anyway. The alternative I am weighing is manual keyset pagination with WHERE id > :cursor: ORDER BY id LIMIT n, which avoids the count and the offset scan but gives up total page numbers and requires a stable indexed sort column.
Is there a documented way to make the QueryBuilder paginator skip or approximate its COUNT query, or does that require a custom adapter?
For a dataset of this size, is accepting the paginator's COUNT plus caching the total a reasonable middle ground, or is keyset the only defensible option?
Does keyset pagination conflict with anything in the paginator result contract (total_items, last, next) that consumers typically rely on?