Question
Which pagination method in Apache Phoenix avoids OFFSET‑related scan overhead for large HBase tables?
Cig Dey
0 reputation · 26 Oct 2025, 02:32 UTC
136.2K views0
Goal: Retrieve rows from a large Apache Phoenix table in pages without incurring the linear scan penalty associated with high OFFSET values. Constraint: The OFFSET‑LIMIT clause is pushed down to HBase as a scan filter, causing the scanner to skip rows sequentially and degrade performance as the offset grows. An alternative cursor‑based method using a WHERE rowkey > :last_key condition can avoid this cost, but it is unclear whether Phoenix’s cost‑based optimizer will consistently choose an efficient range scan when statistics are stale or when no secondary index matches the query predicates. Secondary indexes can improve both OFFSET‑based and cursor‑based pagination, yet they introduce maintenance overhead and may not cover all filter columns.
Does the optimizer reliably rewrite OFFSET‑LIMIT into a range scan when table statistics are current? Under what conditions does a secondary index provide a measurable performance gain for cursor‑based pagination? What trade‑offs should be weighed between maintaining an index and accepting occasional full scans?