429 Too Many Requests error handling in Polygon RPC provider
0 reputation · 11 Jan 2021, 15:10 UTC
0 reputation · 11 Jan 2021, 15:10 UTC
The Polygon RPC layer signals request throttling with an HTTP 429 status and the message "Too Many Requests" once the per‑minute quota is exceeded. When a client library times out or receives a 429, the current pattern is to abort the request and surface a generic error to the caller. There is no standard mechanism for automatic retries or for propagating cancellation across nested async operations.
Should the Polygon RPC layer expose a built‑in exponential‑backoff or retry policy for transient errors such as 429 or network timeouts, or should this responsibility remain entirely with client libraries? The lack of a documented retry strategy leads to inconsistent client behaviour and potential over‑re‑throttling.
35275 reputation · 12 Jan 2021, 01:07 UTC
Polygon’s RPC layer should provide a minimal, configurable retry interface that respects the provider’s rate‑limit boundaries. The provider can expose a simple back‑off policy (e.g., exponential with jitter) and a maximum retry count, while leaving fine‑grained control—such as per‑endpoint rotation or custom jitter—to client libraries.
Retries should only occur for transient failures that are likely to succeed after a short wait. Criteria include:
Retry-After header – honour the specified wait time.Retry-After – apply exponential back‑off starting at 1–2 s, capped at a configured maximum (e.g., 30 s).Retry-After header. If present, wait that number of seconds or until the specified HTTP date.wait = min(maxWait, base * 2**attempt + random(0, 1)) where base is 1 s and maxWait is 30 s.maxRetries (default 3) so that clients can opt‑out of aggressive retrying.retryPolicy: choose between none, exponential, or a custom callback.maxRetries: override the default retry count.backOffBase & backOffMax: adjust the back‑off window.perEndpointLimits: expose per‑endpoint rate limits so clients can rotate endpoints within safe bounds.Client libraries should:
Does your current RPC endpoint return a Retry-After header with every 429? If not, the provider might be using a fixed back‑off window, which could affect how you tune your client’s retry logic.
Providing a lightweight, configurable retry interface in the Polygon RPC layer—respecting Retry-After, using exponential back‑off, and limiting retries—offers the most robust path forward. This approach keeps the provider’s rate‑limit guarantees intact while giving client libraries the flexibility to implement advanced strategies such as endpoint rotation or custom jitter.
Use comments to ask for clarification. Post a solution as an answer.
35,275 reputation · 11 Jan 2021, 16:38 UTC
While implementing a retry policy for 429 errors is essential, relying solely on reactive back-off can still lead to temporary IP bans if multiple concurrent async operations trigger simultaneous retries. A more robust approach involves integrating a client-side request throttler or a token bucket algorithm before the request is even dispatched.
For high-volume dApps, combining a local queue with the Retry-After header logic ensures that retries do not compete with new outgoing requests, which would otherwise exacerbate the throttling cycle. This is particularly critical when using shared public endpoints where quotas are strictly enforced by IP.