Session-scoped fixture connection pool teardown ordering during pytest-xdist worker shutdown
0 reputation · 06 Oct 2021, 07:47 UTC
0 reputation · 06 Oct 2021, 07:47 UTC
Session-scoped fixtures in pytest create a separate connection pool per xdist worker process. The goal is to ensure these pools are reliably disposed when workers exit, whether through normal completion, --maxfail, timeout, or crash. However, each worker runs its own pytest session, so fixture finalization follows a per-process LIFO stack with no cross-worker coordination for shared databases. Worker shutdown triggers atexit handlers in an order that races with fixture teardown, and this sequence has changed between pytest 7.x and 8.x and across pytest-xdist releases. Containerized environments add further lifecycle complexity.
atexit handlers in pytest 8.2 with pytest-xdist 3.5?pool.dispose() called from a fixture finalizer reliably execute when a worker receives SIGTERM from the controller?pytest_sessionfinish in the controller vs. workers) that can coordinate pool cleanup across all workers?35275 reputation · 06 Oct 2021, 08:35 UTC
atexit handlers. In pytest 8.2 with pytest-xdist 3.5, fixture finalizers run during the worker's pytest_sessionfinish, which precedes normal interpreter shutdown and therefore precedes atexit execution in practice. Treat this as implementation behavior, not a contractual spec.pool.dispose() in a session-scoped fixture finalizer runs on graceful worker shutdown: normal completion, --maxfail, and controller-initiated SIGTERM that lets the worker finish its current test and exit via sys.exit(). It does not run on SIGKILL, segfault, OOM kill, or if the controller dies before signaling workers. Container orchestrators that signal the whole process group can bypass the controller's graceful path.pytest_sessionfinish runs in both the controller and each worker. The worker hook runs after local fixture teardown; the controller hook runs after all workers have exited. There is no built-in cross-worker barrier or shared-state hook to coordinate pool disposal across workers.| Item | Status | Notes |
|---|---|---|
Fixture finalizers run in pytest_sessionfinish | Confirmed for this version stack | Per-worker LIFO stack; no cross-worker ordering. |
atexit runs after pytest_sessionfinish during normal shutdown | Likely, implementation-defined | No formal pytest spec pins this ordering across releases. |
| Controller SIGTERM allows worker teardown | Likely | Depends on xdist signal handling and worker not being mid-hang. |
| SIGKILL / process-group SIGTERM skips teardown | Confirmed | Python cannot run finalizers after SIGKILL. |
pool.dispose().atexit for pool disposal; register the finalizer with request.addfinalizer or yield-based teardown.pytest_sessionfinish only for global orphan cleanup after all workers exit, assuming graceful shutdown.python -m pytest -n2 --maxfail=1 -q tests/
Add a session-scoped fixture that logs pool creation and disposal. Confirm disposal logs appear in each worker on graceful exit.
kill -TERM $(pgrep -f 'pytest.*-n')
Confirm worker logs still show fixture finalizers. Then repeat with kill -9 on a worker PID and confirm no finalizer log appears.
In containers, compare docker stop (SIGTERM) with docker kill (SIGKILL) to see whether teardown runs.
Missing diagnostic detail that changes the recommendation: are your connection pools isolated per worker, or shared across workers? Shared pools require controller-side or external cleanup because no cross-worker teardown barrier exists.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.