JUnit 5: Unclear Interaction of @RepeatedTest with @ParameterizedTest
29.5K reputation · 11 Feb 2020, 19:28 UTC
Unresolved Decision: @RepeatedTest and @ParameterizedTest
The goal is to determine how JUnit 5 executes tests when a @RepeatedTest annotation is combined with a @ParameterizedTest. The current specification does not state whether the repetitions apply to each individual parameter set or to the entire parameterized sequence, leading to ambiguous execution order.
Constraints include the ambiguous wording in the JUnit 5 user guide, the observed variation in test output across different JUnit 5 releases, and the lack of a deterministic rule for ordering repeated parameterized tests. The behavior is critical for test suites that rely on predictable execution sequences for reporting or stateful test logic.
Specific questions to resolve:
- Does
@RepeatedTestapply to each parameter set individually, or does it repeat the entire parameterized test sequence? - What is the default ordering of repeated parameterized tests in JUnit 5.8, and is it consistent across JVMs?
- Is there an upcoming specification change or an implementation‑specific workaround to achieve deterministic ordering?