Meteor publication limit and skip pagination with missing server-enforced max
0 reputation · 30 Sept 2023, 07:25 UTC
0 reputation · 30 Sept 2023, 07:25 UTC
Meteor publications are reactive Mongo cursors. The server sends matching documents to client Minimongo unless the publish function explicitly constrains the cursor with limit, skip, fields and a user scoped selector.
Paginating a large collection is developer implemented via limit and skip in publish options or via methods returning pages. There is no framework provided cursor based pagination primitive, and bounding a query relies entirely on the developer to constrain selector, fields and options. The framework does not automatically bound results or enforce row level access control.
An unresolved design decision is how to enforce a server side maximum limit per publication and a per user permission boundary when the framework does not require it. Accidental unbounded publications can sync an entire collection, and reactive skip/limit degrades with large offsets as Mongo must scan skipped documents and reactive update work grows with collection size and change volume.
What server side mechanism should enforce a per publication maximum limit? How should per user permission boundaries be expressed for paginated selectors? Is skip/limit pagination acceptable for large offsets given reactive update cost?
31775 reputation · 30 Sept 2023, 17:31 UTC
To guarantee that a publication never sends more documents than intended, the server must apply a hard cap inside the publish function. The cap is enforced by taking the client‑supplied limit argument, clamping it with a server‑defined MAX constant, and passing the result to cursor.limit(). For user permissions, the selector should always include a filter that ties the data to Meteor.userId() or a role field, ensuring only authorized documents are returned. Finally, for deep paging, replace skip with a range query on a monotonically increasing field such as _id or createdAt to avoid Mongo’s linear scan and the growing cost of reactive updates.
limit values can sync an entire collection, which is a security and performance risk.skip values causes Mongo to scan many documents and the server to push many change notifications.{ _id: { $gt: lastId } }) is more efficient for deep pages.cursor.limit(n), Mongo enforces that limit on the server.Meteor.userId() in the selector is a common pattern to scope data to the logged‑in user.skip values increase the number of change events the server must track, impacting both CPU and network traffic.skip requires and keep the reactive diff small.const MAX_PER_PUB = 1000;Meteor.publish('paginatedItems', function (requestedLimit, lastId) {
check(requestedLimit, Number);
const limit = Math.min(requestedLimit, MAX_PER_PUB);
const selector = { ownerId: Meteor.userId() };
if (lastId) {
selector._id = { $gt: lastId };
}
return Items.find(selector, { sort: { _id: 1 }, limit });
});
_id received and pass it as lastId in the next subscription. This eliminates skip entirely.Meteor.settings.public.debug to log the applied limit and selector. Verify that no subscription ever returns more than MAX_PER_PUB documents.limit parameter in the UI, add a client‑side check to warn when the user requests a value above the cap. The server check is still mandatory.What maximum per publication is appropriate for your application? If you use role‑based access, you may want to add a role field to the selector instead of or in addition to ownerId. Let me know your target cap and any additional permission rules, and I can fine‑tune the example.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.