FastAPI BackgroundTasks: The Smallest Design for Work That Can Wait Until After the Response
FastAPI's BackgroundTasks runs work after the response in the same process. Here's when that's the right call, the scope-leak traps, how to observe it, and when to graduate to a real queue.
11 Jul 2025, 21:37 UTC

You have an endpoint that does its real job in 50 ms, but it also sends a notification email that takes two seconds. Making the client wait for the email is wasteful; standing up Celery, a broker, and worker processes for one email is overkill. FastAPI's BackgroundTasks — a supported feature inherited from Starlette — sits exactly in that gap: it runs a callable after the HTTP response has been sent, inside the same server process. This note lays out when that is the right design, where the trust boundaries are, how to operate it, and the conditions that should push you to a real queue.
Requirements this design actually satisfies
BackgroundTasks is the smallest suitable design when all of the following hold:
- The work is best-effort: losing it on a deploy or crash is acceptable (an audit log line, a cache invalidation, a non-critical webhook).
- The work is short: seconds, not minutes, and shorter than your deploy window.
- You do not need retries, ordering, deduplication, scheduling, or rate limiting.
- You run few enough tasks that sharing the server's event loop and threadpool with request handling is fine.
If any of those fail, skip to the last section — the right answer is an external queue, not a bigger in-process workaround.
The smallest implementation
Declare a parameter of type BackgroundTasks in the path operation, register callables with add_task, and return normally. Starlette awaits the registered tasks after the response bytes have gone to the client.
import logging
from fastapi import BackgroundTasks, FastAPI
log = logging.getLogger("app.tasks")
app = FastAPI()
def send_welcome_email(user_id: int, address: str, correlation_id: str) -> None:
try:
log.info("email start user=%s cid=%s", user_id, correlation_id)
... # call your mail provider here
log.info("email done user=%s cid=%s", user_id, correlation_id)
except Exception:
# The response is already sent; this log is the ONLY place
# the failure will ever surface.
log.exception("email failed user=%s cid=%s", user_id, correlation_id)
@app.post("/users/{user_id}/welcome", status_code=202)
async def welcome(user_id: int, background: BackgroundTasks) -> dict:
user = lookup_user(user_id) # materialize everything NOW
cid = new_correlation_id()
background.add_task(send_welcome_email, user.id, user.email, cid)
return {"status": "queued", "correlation_id": cid}
Execution semantics worth knowing: async def task functions are awaited on the server's event loop; plain def functions are run in Starlette's threadpool. That means a CPU-heavy synchronous task doesn't block the loop directly, but it does consume shared threadpool capacity that other sync endpoints also use.
Trust and data boundaries
The single most common bug with BackgroundTasks is a scope leak: passing objects whose lifetime ends with the request. By the time the task runs, the response is finished and request-scoped resources may be gone.
- Materialize arguments before returning. Pass plain values — IDs, strings, dataclasses. If the task needs a database row, either fetch it before
add_taskand pass the values, or pass the ID and have the task open its own session. - Don't close over the
Requestobject or anything tied to it. Read headers/body in the endpoint and pass what you need as values. - Be careful with yield-based dependencies (the classic
get_db()session generator). The exact ordering of dependency teardown versus background task execution has changed across FastAPI/Starlette versions — do not assume the session is still open inside the task. Confirm the behavior for your pinned versions (a check is described below), or sidestep the question by having the task create its own session. - Keep payloads small. Task arguments live in memory until the task completes. Passing a 200 MB parsed upload as an argument pins that memory; pass an identifier and re-fetch from storage instead.
- No cross-worker coordination. Each worker process behind your load balancer runs its own in-process tasks. Two requests hitting two workers can enqueue duplicate work with no way to see each other.
Operational checks
Because failures are invisible to the client, your only window into background work is your own instrumentation:
- Correlation IDs. Generate one per request, return it in the response, and log it at task start, success, and failure. This is what lets you answer "did the email for request X actually go out?"
- Wrap every task body in try/except. An unhandled exception doesn't hurt the already-sent response, but it produces noise and, worse, can abort any tasks registered after the failing one.
- Watch event-loop lag. A long-running
asynctask that does blocking work (a synchronous HTTP client call, heavy computation) delays every other request in that worker. Monitor event-loop latency or p99 request latency as a proxy.
Two quick verification exercises on a development machine, run with uvicorn app:app (no special permissions needed):
- Add a task that sleeps two seconds and then writes a log line. Call the endpoint with
curland confirm the response returns immediately while the log line appears later. This confirms post-response execution. - Add a task that raises. Confirm the client still gets the success response and the traceback appears only in server logs. This confirms the failure-visibility boundary you're designing your logging around.
- For your pinned FastAPI/Starlette versions, register a task that uses a yield-provided DB session and see whether it works or raises a "session closed" error. The answer determines whether you must open sessions inside tasks.
Failure modes to design around
| Failure | Consequence | Mitigation |
|---|---|---|
| Process restart or deploy | Pending tasks are silently lost | Only use for work where loss is acceptable |
| Exception in task | Client unaffected; error only in logs | try/except + correlated logging |
| Slow async task | Event-loop contention delays requests | Keep tasks short; offload blocking I/O |
| Duplicate requests | Duplicate side effects (two emails) | Idempotency keys at downstream system |
Conditions that change the design
Move to an external queue — Celery, arq, Dramatiq, or a managed cloud queue, with FastAPI only enqueuing jobs — when you need any of: durability (the task must survive restarts), retries with backoff, scheduling (run at 3 AM, run every hour), long-running work beyond a deploy window, or independent scaling of workers. The migration path is usually small: the function you passed to add_task becomes the queue job, and the endpoint swaps background.add_task(fn, ...) for queue.enqueue(fn, ...). Designing your task functions to take plain materialized arguments — which the trust boundaries above already push you toward — makes that swap nearly mechanical.
0 replies
A thoughtful contribution can make all the difference. Be the first to share one.