pytest‑rerunfailures and transactional fixtures: deciding retry scope for test body vs fixtures

Question

When using pytest‑rerunfailures to re‑run flaky tests, the plugin currently executes the whole test function, including all fixture setup and teardown, on each retry attempt.

The goal is to decide whether a retry should preserve fixture state (i.e., re‑run only the test body) or always re‑create fixtures, especially when fixtures perform external writes such as committing rows to a database.

Constraints include the fixture scope (function, class, module, session) which determines whether state is shared between attempts, and the reliance on transactional rollback patterns that assume a clean state after each test execution.

Uncertainty remains about whether the plugin should expose a flag to selectively retry the test body while keeping fixture state, or to keep the current full‑retry behavior, and what the default should be for different fixture scopes.

Should pytest‑rerunfailures add an option to retry only the test body?

How would such an option interact with function‑scoped transactional fixtures versus module‑scoped fixtures that commit data?

What default behavior minimizes duplicate writes without breaking existing test suites that depend on full fixture re‑execution?

Tasadduq Burney · Answer

The Core Conflict: State Isolation vs. Write Efficiency
Adding an option to retry only the test body in pytest-rerunfailures is a high-risk architectural change. While it would reduce duplicate writes in transactional environments, it fundamentally conflicts with the pytest fixture lifecycle, which guarantees that setup and teardown wrap the test execution.

Interaction with Fixture Scopes
The impact of a "body-only" retry varies significantly based on the fixture scope:

Function-scoped Transactional Fixtures: These typically wrap a test in a database transaction and roll it back during teardown. If only the test body is retried, the rollback never occurs between attempts. This leads to accumulated state or duplicate primary key violations if the test body performs commits.
  Module/Session-scoped Fixtures: These are designed to persist across multiple tests. Retrying only the test body would preserve this state, which is generally desired. However, if the fixture itself is the source of the flakiness, a body-only retry will never resolve the issue.

Recommended Implementation Strategy
To minimize duplicate writes without breaking existing suites, the plugin should maintain full-retry as the default and implement an opt-in flag (e.g., --rerun-body-only) with the following logic:

Opt-in Only: Existing suites depend on the guaranteed execution of teardown. Changing the default would cause resource leaks (open sockets, orphaned files) across the ecosystem.
  Scope-Aware Execution: The flag should only bypass function-scoped setup/teardown. Higher-scope fixtures should remain untouched as they are already shared.
  Explicit Warning: Users enabling this mode must be warned that yield fixtures will not execute their teardown code until the final retry attempt (success or failure) completes.

Verification Steps
If implementing or testing this behavior, verify using these scoped checks:
# 1. Verify Function Scope: Ensure teardown is SKIPPED between retries
# 2. Verify Module Scope: Ensure fixture is NOT recreated between retries
# 3. Verify Database: Check that no additional 'INSERT' statements occur 
#    on the second attempt when using a body-only flag.

Missing Diagnostic: To refine this recommendation, it is necessary to know if the "duplicate writes" occur because the fixture commits data during setup, or because the test body commits data that the fixture fails to roll back.

pytest‑rerunfailures and transactional fixtures: deciding retry scope for test body vs fixtures

1 answer

The Core Conflict: State Isolation vs. Write Efficiency

Interaction with Fixture Scopes

Recommended Implementation Strategy

Verification Steps

Your perspective belongs here.

0 question comments