Question
Edge Functions or Serverless Functions for Burst Traffic: Trade‑off Between Concurrency Limits and Latency
Ira Byte
0 reputation · 05 Aug 2022, 01:48 UTC
109.3K views1
Loading…
0 reputation · 05 Aug 2022, 01:48 UTC
When designing a Netlify site that must handle sudden spikes in request volume, engineers need to pick between deploying the logic as an Edge Function or as a traditional Serverless Function. The decision hinges on how each platform’s concurrency limits affect latency when traffic exceeds the allowed simultaneous invocations.
Edge Functions are limited to 100 concurrent invocations per site; excess requests are queued at the edge, which can increase first‑byte latency until a slot frees. Serverless Functions run in a regional Lambda‑backed environment and are constrained by the account’s concurrent execution quota (default 100, adjustable via support); surpassing this quota typically results in HTTP 429 throttling rather than queuing. Because the latency impact of queuing versus throttling differs with request size, client geography, and burst duration, it is unclear which option yields consistently lower response times under sustained overload.
Which function type provides lower average latency when concurrent traffic regularly exceeds the 100‑invocation threshold? Does the geographic advantage of Edge Functions outweigh the queuing penalty compared to the regional latency and possible 429 responses of Serverless Functions under burst conditions?
When the traffic volume regularly exceeds the 100‑invocation limit, Edge Functions usually deliver lower average latency than Serverless Functions, provided the queue stays short enough to keep the first‑byte time within acceptable bounds. If the queue grows beyond the platform’s timeout window or the requests are large and the clients are far from the edge, the latency can exceed that of a regional Serverless Function that simply returns a 429.
To refine the recommendation, I need to know the typical average request payload size and the geographic distribution of your users. These factors influence whether the edge queue or the regional RTT dominates the latency profile.
In most burst scenarios where the edge queue remains short and the payloads are modest, Edge Functions provide a lower average latency due to their lower RTT and avoidance of 429 throttling. If your traffic is dominated by large uploads or your users are far from the edge, a regional Serverless Function may be more reliable, especially if you can raise its concurrency quota to match the burst volume.
Use comments to ask for clarification. Post a solution as an answer.
31,775 reputation · 05 Aug 2022, 05:00 UTC
While the trade-off between queuing (Edge) and throttling (Serverless) is clear, it is important to consider how execution time limits affect the queue's health during a burst. Because Edge Functions typically have much stricter CPU and wall-clock time limits than regional Serverless Functions, a burst of complex requests can lead to a "head-of-line blocking" scenario.
If the 100 concurrent slots are occupied by requests that approach the Edge runtime's timeout limit, the queue depth increases rapidly. This can lead to a cascade where subsequent requests time out while still in the queue, effectively mirroring the 429 behavior of Serverless Functions but with the added penalty of high latency before the failure occurs.
To verify this in a staging environment, it is recommended to:
netlify