Hibernate pagination behavior with fetch joins
0 reputation · 14 Dec 2023, 02:57 UTC
0 reputation · 14 Dec 2023, 02:57 UTC
When implementing pagination in a Java application using Hibernate ORM, the setFirstResult() and setMaxResults() methods are used to limit the result set and define the offset. This ensures the JVM heap is not overwhelmed by large datasets.
However, there is a known architectural conflict when these methods are combined with HQL fetch join operations. If a query fetches a collection of child entities while attempting to paginate the parent entities, the database cannot easily calculate the correct number of root objects due to the Cartesian product created by the join.
This often results in Hibernate retrieving the entire result set into memory to perform pagination manually, which negates the performance benefits of setMaxResults().
35275 reputation · 14 Dec 2023, 05:27 UTC
The most efficient way to paginate root entities while still eagerly loading their collections is to use a two‑step process: first fetch distinct root identifiers with the same joins (but without loading the collections), apply setFirstResult() and setMaxResults() to that identifier query, then fetch the full entities using those identifiers with an IN clause and the desired fetch join.
join fetch on a collection or one‑to‑many association, Hibernate produces duplicate rows for the root entity – one row per matched child.setFirstResult()/setMaxResults() directly on such a query limits the raw row count before duplicates are collapsed, which can return fewer root entities than expected and may force Hibernate to perform pagination in memory.Criteria transformer, but the logical issue remains the same.The identifier‑first query eliminates duplicates because it selects only the root’s primary key (or a distinct set of keys) while keeping the same joins. Limiting this clean set yields the correct page of root IDs. The second query then retrieves the full root entities with the desired fetch join for those IDs, so no duplicate‑row problem affects the limit.
DISTINCT r.id (or the root entity with SELECT DISTINCT r) from the same joins you need for the fetch, but do not include the collection in the select clause.setFirstResult(pageSize * pageNumber) and setMaxResults(pageSize) to this query and execute it to obtain a list of IDs for the current page.FROM RootEntity r LEFT JOIN FETCH r.collection WHERE r.id IN (:ids). Set the ID list as a parameter and execute.This two‑step method works across all major SQL dialects supported by Hibernate (PostgreSQL, MySQL, Oracle, SQL Server, DB2) because it relies only on standard SELECT DISTINCT, LIMIT/OFFSET (or dialect equivalents), and an IN list. The main practical limitation is the maximum number of parameters allowed in an IN clause by some JDBC drivers; if a page could contain more than a few thousand IDs, consider batching the ID list or using a temporary table.
To verify the generated SQL, enable Hibernate logging (show_sql=true or logging.level.org.hibernate.SQL=DEBUG) and confirm that the first query contains a DISTINCT on the root identifier and the appropriate limit/offset, while the second query uses the IN clause with the fetch join.
Missing diagnostic detail that could affect the recommendation: What is the expected maximum number of root entities per page (or total result size)? Knowing this helps decide whether the ID list will stay within driver limits or if batching/temporary tables are required.
Use comments to ask for clarification. Post a solution as an answer.
6,120 reputation · 14 Dec 2023, 14:34 UTC
While the two-step ID retrieval process is a robust solution for collection-based pagination, it is important to clarify that not all fetch join operations trigger in-memory pagination. The performance penalty occurs specifically when fetching collections (@OneToMany or @ManyToMany) because they create a Cartesian product that distorts the root entity count.
In contrast, using fetch join on to-one associations (@ManyToOne or @OneToOne) is generally safe. Since these relationships do not multiply the number of rows returned for a single root entity, Hibernate can still delegate the LIMIT and OFFSET clauses to the database dialect.
To verify if your specific query is triggering in-memory pagination, check your application logs for the following warning:
HHH000104: firstResult/maxResults specified with collection fetch; applying in memory!
If this warning is absent, Hibernate is successfully applying pagination at the SQL level.