Ember Data Store and JSON:API Adapter interoperability for bounded queries and large dataset pagination
0 reputation · 28 Nov 2023, 18:35 UTC
0 reputation · 28 Nov 2023, 18:35 UTC
Goal is to bound server queries and paginate a large dataset without loading the entire collection into the Ember Data identity map. The integration boundary is between the Store and the JSON:API Adapter.
Store.query forwards the query hash to the adapter without enforcing server-side limits. Bounding is expressed via adapter parameters such as page, size, or cursor. Paginated responses are typically interpreted from response meta, links or next cursor. The store does not automatically chain pages or provide a built-in virtual collection, and successive paginated loads merge into the same identity map.
There is an unresolved decision between offset/limit pagination, which can drift under concurrent mutations, and cursor pagination, which requires stable server ordering and explicit client state management for consistency. Behavior of backgroundReload, coalesceFindRequests and identity map merging varies across Ember Data major versions.
What is the expected identity map state when successive bounded store.query calls return overlapping records? How should pagination metadata from meta or links be surfaced to application code without assuming a specific adapter implementation? What client state is required to maintain consistency when choosing cursor pagination over offset/limit with the JSON:API adapter?
34275 reputation · 29 Nov 2023, 06:18 UTC
When successive bounded store.query calls return overlapping records, the Ember Data identity map reconciles them by type and id. Existing record instances are reused, new ids are added, and normalized attributes from the later response are pushed into the already-loaded records. Records absent from a later page are not evicted simply because they were omitted. This behavior is part of the Store contract, but the exact merge semantics for relationships and locally dirty attributes depend on the installed Ember Data major version and serializer; verify against your version before relying on edge-case merge behavior.
To surface pagination metadata without coupling to a specific adapter, preserve the JSON:API document contract in your serializer. In normalizeQueryResponse, keep top-level meta and links so they are accessible as result.meta and result.links on the AdapterPopulatedRecordArray. Treat next, prev, first, last, and any page or cursor meta fields as response data, not adapter internals.
For cursor pagination, the client must own the current cursor or next link, page size, sort/filter parameters, and an invalidation rule for mutations or refresh. Pass the cursor back in the next query hash. The Store does not chain pages or maintain a virtual collection.
Confirmed by the Store contract: store.query forwards the query hash to the adapter and returns an AdapterPopulatedRecordArray. The Store does not inject page, size, or cursor limits and does not automatically fetch subsequent pages.
Likely and version-sensitive: whether attribute updates for already-loaded records replace or merge relationships, and whether locally dirty records are overwritten, vary across Ember Data major versions. backgroundReload and coalesceFindRequests are not pagination controls. backgroundReload may re-run a route query and merge results into the identity map; coalesceFindRequests concerns findRecord coalescing.
backgroundReload semantics.normalizeQueryResponse in your JSON:API serializer to ensure top-level meta and links are preserved.store.query.backgroundReload, decide whether entering the route should re-request the first page or the current cursor, and keep that state outside the query or inside it deterministically.Run a local JSON:API server that returns overlapping ids plus top-level meta and links. Assert that result.meta and result.links are present and that duplicate ids update a single record instance. Re-enter the route with backgroundReload enabled and confirm whether the same cursor or the first page is requested.
What Ember Data major version is installed? The answer changes only if you are on a version where query metadata handling or backgroundReload merge behavior differs materially from the current documented contract.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.