GitLab API pagination mode shift: offset-to-keyset default migration uncertainty
0 reputation · 13 Feb 2020, 00:13 UTC
0 reputation · 13 Feb 2020, 00:13 UTC
GitLab is migrating selected high‑traffic list APIs from offset pagination to keyset pagination, but the decision to make keyset the default across all endpoints and the accompanying timeline are not settled. The goal is to understand whether clients can rely on a consistent default pagination mode and how the shift will affect pagination‑dependent features such as progress indicators and total‑count calculations.
Version‑specific behavior and the differing rollout pace between GitLab SaaS (gitlab.com) and self‑managed instances create uncertainty about which endpoints will adopt keyset pagination first and whether X‑Total headers will continue to be supplied. Clients that depend on X‑Total for page‑count estimation may need to adjust their logic when keyset mode is active, as those headers are omitted.
Will the projects listing endpoint default to keyset pagination in the next GitLab release? How will existing client libraries adapt when X‑Total headers are absent? What migration path should self‑managed operators follow to align with the anticipated SaaS‑style pagination behavior?
34775 reputation · 13 Feb 2020, 05:07 UTC
There is currently no confirmed global timeline for making keyset pagination the default across all GitLab endpoints. Because the migration is handled on a per-endpoint basis to protect database performance, you cannot rely on a consistent default pagination mode across the entire API. Whether the projects listing endpoint defaults to keyset in the next release depends on the specific performance profile of that endpoint in the upcoming version's rollout plan.
When an endpoint switches to keyset pagination, GitLab omits the X-Total and X-Total-Pages headers. This is a deliberate design choice because calculating the total count of a large dataset is computationally expensive and defeats the performance gains of keyset pagination.
Client libraries that rely on these headers for progress indicators or page-count estimations will experience failures or null values. To adapt, clients must shift from count-based logic to existence-based logic:
Total / Per Page to determine the number of requests.next link in the response headers until it is null.To align self-managed instances with SaaS-style behavior and ensure client compatibility, operators should encourage the use of explicit pagination parameters rather than relying on defaults. This prevents breaking changes during version upgrades.
Clients should explicitly request the desired mode to maintain stability:
# To force offset pagination (if still supported by the endpoint)
GET /api/v4/projects?pagination=offset&page=1&per_page=20
# To force keyset pagination
GET /api/v4/projects?pagination=keyset
To verify the current behavior of your specific GitLab version, run the following comparison:
curl --header "PRIVATE-TOKEN: <token>" "https://gitlab.example.com/api/v4/projects?page=1". Note the presence of X-Total.curl --header "PRIVATE-TOKEN: <token>" "https://gitlab.example.com/api/v4/projects?pagination=keyset". Verify that X-Total is absent.Diagnostic Detail Required: Are you using a specific GitLab client library (e.g., python-gitlab, glab), or are you making raw HTTP requests? This determines if the fix requires a library update or a change in your internal request logic.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.