Nova API and Database State: Marker-Based Pagination Consistency
0 reputation · 14 Nov 2024, 18:32 UTC
0 reputation · 14 Nov 2024, 18:32 UTC
The OpenStack Nova API utilizes a marker-based pagination system to retrieve large instance datasets. By combining the limit parameter to bound the result set and the marker parameter to track the cursor, clients can iterate through thousands of servers without triggering gateway timeouts or excessive memory pressure on the API service.
A technical uncertainty exists regarding the behavior of this cursor when the underlying database state changes during a multi-request retrieval sequence. Because the marker tracks a specific point in the dataset rather than a static snapshot, modifications to the resource list can impact the sequence of returned items.
How does the Nova API handle the consistency of the result set when resources are added or removed between paginated requests? Does the marker implementation guarantee that every existing resource is seen exactly once during a full traversal?
33275 reputation · 15 Nov 2024, 04:11 UTC
No. Nova's marker-based pagination for GET /servers behaves as a live cursor over the current database state, not as a snapshot. It does not guarantee that every resource present at the start of a traversal will be seen exactly once if instances are created, deleted, or updated between requests.
The marker is a cursor value evaluated against a sort key on each request. The API applies a keyset-style predicate rather than an OFFSET calculation, so each page reflects the rows that satisfy the predicate at query time. The following effects are the expected consequences of that model, though the exact default sort key and accepted parameters vary by Nova version and API microversion.
limit and marker, then diff the returned UUIDs page by page.Marker semantics, the default sort key and direction, and whether sort_key and sort_dir are accepted depend on the Nova release and API microversion. If your deployment spans multiple cells or read replicas, cursor stability may vary further. Treat the marker as a cursor, not a snapshot token, and confirm the generated query uses a keyset predicate rather than OFFSET before relying on pagination for incident analysis.
If this question relates to a specific incident, provide the exact Nova version and API microversion together with the sort_key and sort_dir in use. Without those, only the general consistency model above can be offered.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.