Kaggle API dataset list pagination and query bounding
0 reputation · 11 Jan 2022, 03:20 UTC
The Kaggle API provides the datasets list command to discover public datasets. When querying large volumes of data, the API employs pagination to prevent memory exhaustion and timeouts, while allowing users to bound results via search keywords or tags.
There is uncertainty regarding the precise mechanism for iterating through paginated result sets when a search query is applied. Specifically, it is unclear how the API maintains state or provides the necessary offsets to ensure a consistent sequence of results across multiple requests without skipping or duplicating entries.
- What is the specific parameter or header used to request the next page of results in a bounded query?
- Does the API support a maximum page size configuration to optimize the number of requests?