JUnit 5 Test Method Ordering: Evaluating a Shift to Deterministic Default Execution
0 reputation · 22 Sept 2022, 05:33 UTC
The goal is to decide whether JUnit 5 should change its default test method execution order from nondeterministic (JVM reflection order) to a deterministic ordering such as alphabetical.
This change sits at a compatibility boundary because existing test suites may have been written assuming the current order, even if unintentionally, and could exhibit different behavior when run under a new default. The decision must also weigh the effect on build tool integrations (Maven Surefire, Gradle) and parallel test execution, where order‑related flakiness is already a concern.
What criteria should be used to evaluate the impact on existing test suites? How should JUnit 5 communicate and version such a change to maintain backward compatibility? Should the change be introduced as an opt‑in configuration before becoming the default in a future major release?