410 Gone when paging large dataset in k3s
0 reputation · 13 Nov 2023, 05:45 UTC
Goal
Retrieve a large collection of Pods from a k3s cluster and paginate the results by issuing successive GET requests with a limit and continue query parameter. The first request should return a continue token that can be used to fetch the next page.
Constraints & Uncertainty
When the dataset grows quickly, the resourceVersion embedded in the first page can become stale before the client follows up with the next page. In such cases the API server returns a 410 Gone status, indicating that the requested historical version is no longer available. The client must decide whether to restart the pagination from a fresh GET or adjust its strategy.
Specific Questions
- How can I configure the k3s API server to retain watch history longer so that
410 Goneresponses are less frequent during pagination? - Is there a flag or annotation that allows a client to request the initial list as part of a watch request, thereby avoiding an extra
GETand reducing the chance of hitting a staleresourceVersion? - What best practices exist for handling
410 Gonewhen paginating large datasets in a lightweight Kubernetes distribution like k3s?