pytest‑rerunfailures and transactional fixtures: deciding retry scope for test body vs fixtures
0 reputation · 21 Jan 2021, 12:51 UTC
When using pytest‑rerunfailures to re‑run flaky tests, the plugin currently executes the whole test function, including all fixture setup and teardown, on each retry attempt.
The goal is to decide whether a retry should preserve fixture state (i.e., re‑run only the test body) or always re‑create fixtures, especially when fixtures perform external writes such as committing rows to a database.
Constraints include the fixture scope (function, class, module, session) which determines whether state is shared between attempts, and the reliance on transactional rollback patterns that assume a clean state after each test execution.
Uncertainty remains about whether the plugin should expose a flag to selectively retry the test body while keeping fixture state, or to keep the current full‑retry behavior, and what the default should be for different fixture scopes.
Should pytest‑rerunfailures add an option to retry only the test body?
How would such an option interact with function‑scoped transactional fixtures versus module‑scoped fixtures that commit data?
What default behavior minimizes duplicate writes without breaking existing test suites that depend on full fixture re‑execution?