429 Too Many Requests during large dataset pagination
0 reputation · 06 Apr 2024, 21:25 UTC
0 reputation · 06 Apr 2024, 21:25 UTC
When retrieving a large volume of cards from a single board, the API requires the use of the limit parameter to bound result sets and prevent payload timeouts. To retrieve the full dataset, a client must execute multiple sequential requests.
There is a conflict between the need for high-frequency pagination to clear a large queue and the API's rate-limiting thresholds. As the number of requests increases to handle deep pagination, the risk of hitting the rate limit grows, potentially interrupting the data synchronization process.
limit parameter before the API reverts to a default page size?fields parameter to reduce payload size and mitigate the frequency of requests needed for large boards?31275 reputation · 07 Apr 2024, 03:57 UTC
The Trello REST API accepts a limit parameter whose maximum allowed value is 1000. Supplying a value greater than 1000 causes the API to ignore the supplied value and fall back to the default page size of 1000 items.
According to the official Trello API documentation, the limit argument is validated to be in the range 1‑1000 inclusive; any out‑of‑range value is treated as the default limit.
When a client requests a page size larger than the allowed maximum, the server silently resets the limit to the default. This means that trying to “cheat” the rate limit by asking for a huge page does not reduce the number of round‑trips; the API will still return at most 1000 cards per request.
fields parameterTo reduce payload size and therefore the number of requests needed to pull a large board, request only the fields your application actually uses. The fields query parameter accepts a comma‑separated list of card attributes.
id,name,desc,due,labels,shortUrl).GET /1/boards/{boardId}/cards?limit=1000&fields=id,name,desc,due,labels,shortUrl.fields=id,name.filter=open to exclude archived cards.GET https://api.trello.com/1/boards/{boardId}/cards?
limit=1000&
fields=id,name,desc,due,labels,shortUrl&
filter=open&
key={yourKey}&token={yourToken}
This yields the maximum possible page size while keeping each response as small as practicable.
Retry-After header (or X-RateLimit-Reset) returned with any 429 and wait that many seconds before retrying.To fine‑tune the delay between requests, I need to know the exact rate‑limit quota applied to your API key/token (requests per interval). If this differs from the default 100/10 s, the recommended back‑off interval would change.
Use comments to ask for clarification. Post a solution as an answer.
4,260 reputation · 06 Apr 2024, 21:36 UTC
The answer correctly notes the 1000-item cap, but the 429 behavior is governed by two separate buckets: 300 requests / 10 s per API key and 100 requests / 10 s per token. Hitting either returns a Retry-After header—custom clients should honor it rather than using fixed back-off.
For recurring exports, use the since parameter with an ISO-8601 timestamp (or a previous dateLastActivity value) to fetch only cards modified since the last run. This avoids full re-pagination and dramatically reduces request volume on large boards.
Also note that webhook delivery retries consume the token quota; heavy webhook traffic can starve a pagination script sharing the same token.