Firebird OFFSET/FETCH pagination and JDBC driver handling of large offsets
0 reputation · 29 Sept 2023, 15:10 UTC
0 reputation · 29 Sept 2023, 15:10 UTC
Goal: Retrieve a consistent page of data from a large Firebird table using the OFFSET/FETCH clause through a JDBC connection, while maintaining deterministic order and acceptable response time.
Constraints: The OFFSET/FETCH syntax requires an explicit ORDER BY to guarantee stable pagination; without it the server may return rows in an undefined order, making pages non‑deterministic. Large offset values force the engine to scan and discard preceding rows, which can degrade performance. The JDBC driver must implement Firebird 3.0+ SQL grammar to parse OFFSET/FETCH correctly, especially when the offset and fetch values are supplied as parameter markers.
Uncertainty: It is unclear whether the driver treats parameter‑bound offset/fetch values as literal values in the SQL statement, whether it adds any client‑side buffering that could alter the semantics of OFFSET/FETCH, and how the combination of OFFSET/FETCH with ORDER BY behaves when the underlying data changes between fetches.
Specific questions:
31775 reputation · 29 Sept 2023, 19:50 UTC
Yes, Jaybird accepts OFFSET and FETCH values as JDBC parameters; no, row order is not guaranteed without ORDER BY; and no, the driver does not buffer the whole result set — the server scans and discards the offset rows itself, so large offsets cost server-side work proportional to the offset, not client memory.
Firebird 3.0+ implements the SQL-standard OFFSET n ROWS FETCH NEXT m ROWS ONLY syntax, and both the offset and fetch expressions accept parameter markers. With Jaybird 3.x or later (4.x/5.x recommended for Firebird 4.0/5.0), a prepared statement like this works:
String sql = "SELECT id, name FROM items "
+ "ORDER BY id "
+ "OFFSET ? ROWS FETCH NEXT ? ROWS ONLY";
try (PreparedStatement ps = conn.prepareStatement(sql)) {
ps.setInt(1, offset);
ps.setInt(2, pageSize);
ResultSet rs = ps.executeQuery();
}Jaybird forwards the statement to the engine unchanged; it does not rewrite parameters into literals or emulate the clause client-side. Two caveats: this only works with the standard syntax (Firebird's older FIRST/SKIP or ROWS variants have their own rules), and Jaybird 2.2.x targeting Firebird 2.5 does not support OFFSET/FETCH at all — check your driver version before assuming a syntax error comes from the SQL.
Without ORDER BY, the result sequence is genuinely undefined. Firebird returns rows in whatever order the chosen access path produces — physical storage order, index order, or something else depending on the plan. It is not "usually insertion order" in any reliable sense: plan changes, garbage collection, or index availability can alter the sequence between executions. For pagination this means pages can overlap or skip rows even on a static table. Always include ORDER BY, and make it deterministic by ending with a unique tiebreaker such as the primary key.
The engine evaluates OFFSET on the server by producing the ordered result and discarding the first n rows. Streaming to the client begins only after the skip, so Jaybird never materializes the skipped rows — there is no hidden client-side buffering of the full result set. The cost is server-side I/O and CPU roughly proportional to the offset, plus the sort cost if ORDER BY lacks a supporting index. OFFSET 100 000 on a million-row table with an indexed ORDER BY is noticeably slower than OFFSET 0; with an unindexed sort it can be much worse. The driver's fetch size (setFetchSize or the useFbFetchSize connection property) only tunes network round-trips for the rows actually returned — it does not reduce the offset scan.
Each page query reflects the database state when it executes. If rows are inserted, deleted, or reordered between fetches, offset-based pages can miss or duplicate rows. Two mitigations:
WHERE id > :lastSeen ORDER BY id FETCH NEXT ? ROWS ONLY — which is both stable under concurrent change and avoids the offset scan entirely.To confirm behavior on your setup: run the parameterized query above against your server and confirm no syntax error; execute the same query twice without ORDER BY and compare sequences; and time OFFSET 0 versus OFFSET 100000 with SET PLAN ON (or isql's plan output) to confirm the ORDER BY uses an index rather than a full sort. Enabling Jaybird logging at FINE level for org.firebirdsql.jdbc will show the statement is sent unchanged.
One open detail that would sharpen the performance advice: the exact Firebird server and Jaybird versions in use, since plan behavior and driver defaults differ across releases.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.