FastAPI endpoint timeout and client-disconnect cancellation: choosing a consistent strategy without a built-in setting
0 reputation · 17 Mar 2021, 03:17 UTC
0 reputation · 17 Mar 2021, 03:17 UTC
I have several long-running async endpoints in a FastAPI application (assume FastAPI 0.1xx on Starlette, served by Uvicorn) and I want requests to stop doing work when either a client disconnects or a time budget is exceeded. My understanding is that FastAPI exposes no per-request timeout setting, so the behavior has to come from somewhere else.
The pieces I am aware of are: Starlette's request.is_disconnected(), which appears to require explicit polling inside the endpoint; wrapping the handler logic in asyncio.wait_for, which raises asyncio.TimeoutError that I would need to catch and map to a response; and server- or proxy-level timeouts in Uvicorn or a reverse proxy. I am unsure how these interact, and whether cancellation actually propagates into awaited downstream coroutines (database calls, HTTP clients) in each case.
There is also a boundary I cannot resolve: BackgroundTasks seem to run after the response and are not tied to disconnects, so it is unclear whether they should be excluded from any cancellation policy entirely.
Which layer (endpoint wrapper, dependency, middleware, or ASGI server) is the appropriate place to enforce a per-route time budget? Does polling is_disconnected() reliably fire on Uvicorn while the coroutine is awaiting I/O, or only after it yields? And should background tasks be treated as outside the request lifecycle for cancellation purposes?
33275 reputation · 17 Mar 2021, 10:46 UTC
For a per-route time budget, wrap the endpoint logic in asyncio.wait_for at the endpoint or dependency level. For client disconnects, rely on the ASGI server's task cancellation rather than polling request.is_disconnected() inside long awaits. Treat BackgroundTasks as outside the request lifecycle and exclude them from both policies.
FastAPI does not expose a built-in timeout setting. The most precise place is the endpoint itself or a route-scoped dependency. Middleware can wrap the whole application, but it is coarse-grained and harder to tune per route.
Use asyncio.wait_for and catch asyncio.TimeoutError explicitly. Without a custom exception handler, the timeout surfaces as an unhandled exception and returns HTTP 500.
import asyncio
from fastapi import FastAPI, HTTPException
app = FastAPI()
@app.get("/slow")
async def slow():
try:
return await asyncio.wait_for(do_work(), timeout=5.0)
except asyncio.TimeoutError:
raise HTTPException(status_code=504, detail="Time budget exceeded")
Assumption: FastAPI 0.1xx on Starlette with Uvicorn. The exact exception type and handler registration may vary slightly across versions, so verify against your pinned release.
request.is_disconnected() is a polling mechanism, not an interrupt. It reads from the ASGI receive channel. If your coroutine is blocked in a single long await, the method will not fire until the event loop regains control and you explicitly poll it. It is useful only for cooperative checkpoints inside loops that yield regularly.
Uvicorn and other ASGI servers generally cancel the request task when the client disconnects. This is the more reliable path for true disconnects, provided the task is awaiting cancellable I/O. A blocking synchronous call will ignore both server cancellation and timeout signals.
Yes, if the downstream code is properly awaited. asyncio cancellation propagates through awaited coroutines and raises CancelledError at the current await point. If a database driver or HTTP client blocks the event loop synchronously, cancellation cannot interrupt it. Confirm that your clients are async-native.
BackgroundTasks execute after the response is sent. They are detached from the request-response lifecycle. A client disconnect or a wait_for timeout in the endpoint will not cancel them. Treat them as a separate lifecycle with their own timeout and supervision strategy.
await asyncio.sleep(30). Close the client connection mid-request and check server logs for task cancellation.asyncio.wait_for and confirm your exception handler returns 504, not 500.BackgroundTask and disconnect before it finishes; verify it still runs to completion.Are your downstream database and HTTP calls truly asynchronous, or do any of them block the event loop? If blocking calls exist, neither wait_for nor server disconnect cancellation will interrupt them, and you will need to move that work to a thread or process pool before any timeout strategy is effective.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.