Mongoose list endpoint is slow: how to tell hydration overhead from a bad query plan before adopting lean()
0 reputation · 24 Oct 2024, 06:57 UTC
0 reputation · 24 Oct 2024, 06:57 UTC
On a read-heavy Mongoose list endpoint (assume Mongoose 7/8, Node 18+, MongoDB 6), the plan is to measure before optimizing. The candidate lever is the documented query.lean() option, which returns plain JavaScript objects instead of hydrated documents, skipping change tracking, getters, and middleware state. But that only helps if request time is dominated by client-side document instantiation rather than server-side work.
The uncertainty is twofold. First, how to attribute latency correctly: explain() on the underlying MongoDB query shows index usage and collection scans, while a benchmark timing the same query with and without .lean() at realistic result sizes shows hydration cost — but it is unclear which measurement should gate the decision, and at what result-set size lean() typically becomes worthwhile. Second, the adoption strategy: enabling lean() per query versus globally via a plugin changes the contract for every downstream consumer, since lean results lose instance methods, virtuals, and save().
What is a reliable way to separate hydration overhead from query-plan problems in profiling? Is there a documented or conventional threshold for when lean() pays off? And is global lean-by-default considered safe practice, or should it stay opt-in per query?
31275 reputation · 24 Oct 2024, 07:22 UTC
await MyModel.collection.find(filter)
.sort(sort)
.limit(limit)
.explain();
Look at executionStats.executionTimeMillis and the ratio totalDocsExamined / totalDocsReturned. A high ratio (>10) or a collection scan (stage: "COLLSCAN") indicates a query‑plan problem that .lean() cannot fix.
await query.exec() part, once with .lean() and once without, keeping everything else identical:const t0 = process.hrtime.bigint();
const docsLean = await MyModel.find(filter).lean().limit(limit).exec();
const t1 = process.hrtime.bigint();
const leanTime = Number(t1 - t0) / 1e6; // ms
const t2 = process.hrtime.bigint();
const docsFull = await MyModel.find(filter).limit(limit).exec();
const t3 = process.hrtime.bigint();
const fullTime = Number(t3 - t2) / 1e6; // ms
console.log({ leanTime, fullTime, delta: fullTime - leanTime });
The difference approximates client‑side hydration cost (object instantiation, getters, virtuals, middleware). Run the pair several times and average to reduce noise.
.lean() will not help..lean() is a worthwhile optimization..lean() pays offBased on observed behavior with medium‑sized documents (~1‑2 KB each):
These numbers are guidelines; the only reliable way to decide for your specific schema and payload is to run the benchmark described above.
lean() – safe or opt‑in?Enabling lean() globally is safe only when **no** downstream code relies on:
.save(), .remove(), .populate()).
this.isModified).
If any of those are used, global lean will break contracts and cause silent failures or runtime errors. In that case keep lean() opt‑in per query or scope it with a plugin that applies only to selected routes.
Missing diagnostic detail that changes the recommendation: Do any of your endpoint consumers (controllers, services, or client‑side code) call Mongoose document methods, access virtuals, or depend on getters/setters? If the answer is yes, global lean is not advisable; if no, you may consider a global lean‑by‑default plugin after running the full test suite.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.