ShotGrid REST API Pagination and Data Consistency

Question

Entity Retrieval via REST API
When synchronizing large datasets from ShotGrid (Autodesk Flow Production Tracking), the REST API requires pagination to manage result sets and avoid server timeouts. The system utilizes tokens or offset markers to navigate through these pages of entity data.

Concurrent Update Constraints
A challenge arises when entities are modified or created while a paginated read sequence is in progress. Because the API must maintain a stable result set across multiple requests, there is uncertainty regarding how the server handles shifts in the underlying data index during active pagination.

Potential for duplicate records if an entity moves to a subsequent page.
  Risk of skipping records if an entity is deleted or moved to a previous page.

What is the documented behavior for maintaining result set integrity during concurrent updates? Does the API provide a snapshot-based mechanism to ensure a consistent view of the data across all pagination tokens?

Tasadduq Burney · Answer

Documented behavior
The ShotGrid REST API uses cursor‑based pagination and does not provide a snapshot‑locked view; tokens may become stale if entities are created, deleted, or re‑indexed while paging, which can cause duplicate or missing records. The API does not expose a query‑ID or snapshot flag to freeze the data set.

Likely Explanation
The API treats each token as a pointer to the current position in the result set at the moment the request is issued. Tokens are considered valid only for the duration of the pagination session, and there is no explicit parameter to create an immutable query context.

Confirmed Facts

The endpoint relies on cursor tokens rather than offset numbers.
Tokens are valid while the client continues the same pagination sequence.
The official reference does not mention a query‑ID, snapshot flag, or any mechanism to create a stable query context.
Response metadata does not include a version stamp that guarantees consistency.

Steps to Verify Stability in Your Use Case

Perform an initial GET request and capture the returned token.
Continue issuing subsequent GET requests using the same token until the final page is retrieved; do not request a new token until the previous page has been fully processed.
Log the entity IDs returned on each page and compare them across pages for duplicates or gaps while you simulate concurrent updates (e.g., create or delete an entity via the API).
If inconsistencies appear, consider pausing updates during the pagination window or fetching the entire data set in a single request if the API supports a “full export” endpoint.

Missing Diagnostic Detail
Please confirm whether ShotGrid provides a query‑ID or similar parameter that can be reused to create a stable, snapshot‑like context for pagination.

ShotGrid REST API Pagination and Data Consistency

Entity Retrieval via REST API

Concurrent Update Constraints

1 answer

Documented behavior

Likely Explanation

Confirmed Facts

Steps to Verify Stability in Your Use Case

Missing Diagnostic Detail

Your perspective belongs here.

0 question comments