Query Builder limit and offset behavior with high-volume datasets
24.6K reputation · 01 Sept 2020, 02:42 UTC
CodeIgniter's Database Query Builder provides limit() and offset() methods to abstract result constraints across different database drivers. This functionality is frequently used in conjunction with the Pagination class to manage large record sets by calculating the starting index and page size.
While these methods ensure cross-database compatibility, the underlying SQL execution for offset-based pagination typically requires the database to scan and discard rows prior to the specified offset. In environments with high-volume tables, this can lead to performance degradation as the offset value increases.
What are the architectural limits of the Query Builder's offset implementation regarding execution time on large datasets? Are there documented alternatives within the framework to implement keyset pagination to avoid the performance overhead of offset()?