Sequelize limit behavior with one-to-many eager loading
26.5K reputation · 10 Jul 2023, 22:13 UTC
When implementing pagination in Sequelize, the limit and offset options are used to control the number of records returned from the database. However, a specific behavior occurs when these options are combined with the include property for one-to-many associations.
Because Sequelize generates SQL JOINs for eager loading, the resulting row count includes the joined child records. This often causes the ORM to return fewer parent records than the value specified in the limit option, as the limit is applied to the total number of joined rows rather than the unique parent entities.
While the separate: true option is documented as a way to handle this by executing queries in distinct steps, it changes the execution pattern of the data retrieval.
- Under what specific conditions does Sequelize automatically handle the join-induced limit truncation without requiring
separate: true? - How does the performance overhead of
separate: truecompare to using a subquery for pagination on large datasets?
1 answer
1 question comment
Use comments to ask for clarification. Post a solution as an answer.
26,525 reputation · 11 Jul 2023, 05:22 UTC
The automatic subquery that Sequelize applies to respect limit/offset with eager loading relies on the primary model’s primary key being present in the selected columns. By default, attributes is undefined, so all columns—including the primary key—are selected, allowing Sequelize to deduplicate joined rows correctly. If you explicitly set attributes and omit the primary key (or any unique combination that identifies a parent row), Sequelize cannot guarantee unique parent IDs in the subquery result and may either fall back to a flat join (producing fewer parents than the limit) or generate a separate query for the association. This behavior is consistent across MySQL, PostgreSQL, and SQLite, but for MSSQL the subQuery flag defaults to false in some versions, requiring you to set it to true explicitly to enable the subquery approach.