dbplyr LIMIT translation and DBI chunked fetch for large remote tables
0 reputation · 15 Sept 2026, 20:12 UTC
Goal is to bound a query and paginate a large remote dataset without materializing the full result in R.
dbplyr documents that slice_head(n) and explicit limit() are rendered as SQL LIMIT by the backend dialect for remote tables. DBI documents a streaming fetch model via dbSendQuery followed by dbFetch(n = ...), with fetch size controlling how many rows are pulled per round-trip.
Uncertainty remains about how these mechanisms interact and where backend variance applies. Pagination in dbplyr is implemented via LIMIT/OFFSET semantics with no universal cursor abstraction in core dbplyr, and fetch size support and honoring is driver-specific.
Does dbplyr's translated LIMIT guarantee that DBI will never attempt to fetch more rows than the limit? Is fetch size respected when a dbplyr remote table is collected via dplyr verbs? When using LIMIT/OFFSET pagination, does DBI streaming apply per page or across the whole statement?