Twitter API v2 filtered stream: total timeout or keep-alive idle detection for reconnects?
0 reputation · 13 Jan 2021, 21:43 UTC
0 reputation · 13 Jan 2021, 21:43 UTC
A client consuming the filtered stream endpoint (GET /2/tweets/search/stream) holds a long-lived HTTP connection the platform can drop at any time; documented guidance is to reconnect with exponential backoff and to wait longer after HTTP 429 responses using headers such as x-rate-limit-reset.
The unresolved decision is the reconnect trigger: a hard total connection lifetime that forces periodic reconnects even on a healthy stream, versus idle detection that treats an absence of tweets and keep-alive signals beyond a threshold as the drop signal. Forced reconnects add churn; idle-only detection risks a stalled connection persisting if keep-alive cadence is misjudged.
Constraints: cancellation should abort the underlying request cleanly (for example via AbortController or context cancellation) rather than leave the stream hanging; rules managed through POST /2/tweets/search/stream/rules persist independently, so reconnecting does not require re-adding them; and keep-alive intervals, timeout values, and streaming availability vary by API version and access tier, so specific numbers need checking against current documentation.
35275 reputation · 14 Jan 2021, 02:55 UTC
Twitter's v2 filtered-stream endpoint does not impose a documented fixed maximum connection lifetime. Clients should rely on idle detection via keep-alive signals rather than a bounded connection lifetime. The platform sends keep-alive newline packets (typically every ~30 seconds); a connection should be treated as dropped only when the gap between these signals exceeds a defined threshold (commonly 30–60 seconds).
Because stream rules persist server-side via POST /2/tweets/search/stream/rules, reconnecting does not require re-configuring your filters. This removes the primary penalty of frequent reconnects, but forced periodic churn is still discouraged as it increases the risk of hitting rate limits during the handshake phase.
AbortController in JavaScript or context.CancelFunc in Go). This prevents leaked sockets and ensures the underlying HTTP request is terminated cleanly.curl -v or a raw socket listener to confirm that a newline character is received approximately every 30 seconds during periods of zero tweet activity.AbortController or context cancellation while the stream is active and verify the socket closes immediately.x-rate-limit-reset header if an HTTP 429 is encountered.Keep-alive intervals and implicit server-side idle timeouts can vary by API access tier. While ~30 seconds is the standard observation, these values are not strictly guaranteed across all tiers. If you are using a specific Enterprise or Premium tier, verify the exact cadence in your tier's documentation.
To provide a more precise idle threshold recommendation, please specify your API access tier (e.g., Free, Basic, Pro, or Enterprise), as timeout behaviors may differ by tier.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.