Nova API and Database State: Marker-Based Pagination Consistency
0 reputation · 14 Nov 2024, 18:32 UTC
Dataset Stability During Paged Retrieval
The OpenStack Nova API utilizes a marker-based pagination system to retrieve large instance datasets. By combining the limit parameter to bound the result set and the marker parameter to track the cursor, clients can iterate through thousands of servers without triggering gateway timeouts or excessive memory pressure on the API service.
A technical uncertainty exists regarding the behavior of this cursor when the underlying database state changes during a multi-request retrieval sequence. Because the marker tracks a specific point in the dataset rather than a static snapshot, modifications to the resource list can impact the sequence of returned items.
- If an instance is deleted from a page already processed, the marker's relative position may shift.
- If new instances are created that sort before the current marker, they may be omitted from the current iteration.
How does the Nova API handle the consistency of the result set when resources are added or removed between paginated requests? Does the marker implementation guarantee that every existing resource is seen exactly once during a full traversal?