Linear Scan Latency in RocksDB Iterator Pagination

Implementing pagination in RocksDB requires using the rocksdb::Iterator interface to manually bound result sets. Since the engine lacks native LIMIT or OFFSET clauses, applications typically use Seek() to jump to a known key and call Next() for a specific count of records.

When paginating through very large datasets, there is a potential performance degradation as the offset increases. Because the iterator must linearly scan from the last known key to reach the desired page, the time complexity for retrieving deep pages may increase significantly.

Using a rocksdb::Snapshot ensures a consistent view of the data across pages, but this introduces additional memory overhead and may impact the compaction of old SST files.

• What is the expected performance impact on read latency when performing deep pagination via linear iterator scans?
• Are there documented strategies to avoid linear scanning when retrieving distant pages in a large dataset?