Paginated ORM query with joinedload returns wrong page sizes and slow total count
0 reputation · 28 Feb 2026, 02:58 UTC
0 reputation · 28 Feb 2026, 02:58 UTC
A server-side paginated listing built on SQLAlchemy ORM (2.0-style select() with Session.execute()) needs to show page numbers plus a total result count. The query applies limit()/offset() and also eager-loads a one-to-many collection with joinedload().
With SQL echoing enabled, the generated statement wraps everything in a subquery before applying LIMIT/OFFSET, which appears to come from the joined eager load of the collection. Separately, the COUNT query used for the page-number UI runs over the same filtered joins and costs roughly as much as the page fetch itself, doubling the load per request.
The UI requires jumping to arbitrary page numbers, so a pure keyset/seek approach may not fit. The collection must be eager-loaded to avoid N+1 queries. Assume SQLAlchemy 2.0 on PostgreSQL, with a unique ORDER BY including the primary key already in place.
1. Should the joinedload() be switched to selectinload() so the limit applies cleanly to parent rows, and what trade-offs does that introduce?
2. What is the recommended pattern for an efficient total count alongside a paginated query when the filter involves joins?
3. Is the subquery wrapping with joinedload plus limit/offset guaranteed behavior in 2.0, or loader-strategy dependent?
Yes: switch the collection to selectinload(). The subquery wrapping you see is expected behavior for joinedload() of a one-to-many collection combined with limit()/offset() — SQLAlchemy wraps the query so the limit applies to distinct parent rows instead of the row-multiplied join result. It is loader-strategy dependent, not universal: joinedload() of a many-to-one does not trigger it, and selectinload() never does.
With selectinload() the parent query becomes a flat SELECT ... LIMIT ... OFFSET ..., and children load in a second query with WHERE parent_id IN (...). Page sizes become exact and the parent query plans predictably against your unique ORDER BY.
from sqlalchemy import select
from sqlalchemy.orm import selectinload
stmt = (
select(Parent)
.options(selectinload(Parent.children))
.where(Parent.status == "active")
.order_by(Parent.created_at, Parent.id)
.limit(page_size)
.offset((page - 1) * page_size)
)Trade-offs: two round trips instead of one, and a large IN list if pages are wide — keep page sizes bounded (e.g., ≤ 100). For paginated listings this is almost always the right exchange. An alternative hybrid — fetch parent IDs with limit/offset first, then a joinedload query filtered by those IDs — keeps one join query but adds a round trip and complexity, so it only pays off when the join itself is cheap and pages are large.
Never count from the eager-loaded query: the collection join inflates rows and forces a DISTINCT or subquery. Build a separate minimal statement with no eager loads, ordering, limit, or offset:
from sqlalchemy import func, select
count_stmt = (
select(func.count())
.select_from(Parent)
.where(Parent.status == "active")
)
total = session.scalar(count_stmt)If the filter joins are many-to-one only, they don't change the parent count, so join just the tables the WHERE clause needs. If a filter touches a one-to-many, prefer WHERE EXISTS over a join plus count(distinct Parent.id) — PostgreSQL can use a semi-join plan and avoid scanning the many side:
from sqlalchemy import exists
count_stmt = select(func.count()).select_from(Parent).where(
Parent.status == "active",
exists().where(Child.parent_id == Parent.id, Child.flag.is_(True)),
)It is deterministic for the specific combination — joined eager loading of a collection (one-to-many or many-to-many) plus limit/offset — in SQLAlchemy 2.0. It is not a property of eager loading in general, which is why the behavior appears to change when you alter loader options.
before_cursor_execute listener: with selectinload you should see a flat parent SELECT plus a second IN query, no subquery.ORDER BY, LIMIT, OFFSET, or join to the collection table unless the filter requires it.EXPLAIN ANALYZE on both queries against production-like data to confirm the count stops scanning the child table.Assumes SQLAlchemy 2.0 on PostgreSQL as stated; the loader behavior described is stable across the 2.0 series, but confirm the emitted SQL on your exact version before shipping.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.