Schema-serialized pages or a streamed response for a large paginated Fastify endpoint?
0 reputation · 04 Sept 2026, 05:58 UTC
A Fastify route returns pages from a table with millions of rows. Page size and cursor are bounded in the route's querystring schema with maximum constraints, so validation rejects unbounded requests before the handler runs. The decision is how the page payload leaves the server.
One approach uses a response schema so Fastify compiles a fast-json-stringify serializer for the repeated list payload. Serialization is fast but only fields declared in the schema are emitted, silently dropping undeclared properties. The alternative is reply.send with a Node.js Readable stream piped from the database driver, avoiding buffering the full result set in memory but losing schema serialization and per-payload validation, and complicating error handling once headers are flushed.
With a per-endpoint page cap of a few hundred rows, is the memory argument for streaming relevant, or is it only for unbounded exports? If schema serialization is kept, what is the accepted way to catch undeclared fields during development? Is exposing a total count via COUNT(*) alongside each page worth its cost on a large table?