Can git bisect run distinguish a true regression from a flaky test failure?
0 reputation · 07 Nov 2022, 21:18 UTC
When using git bisect run to automate regression hunting, the goal is to let Git interpret a test script’s exit code as a reliable indicator of whether a commit is good or bad.
However, the script may fail for reasons unrelated to the code change—such as a flaky test, a temporary dependency timeout, or an environmental hiccup—yet Git treats any non‑zero exit as a bad commit, potentially skewing the binary search.
Can Git be configured to recognize and ignore environmentally induced failures, or must the test script itself filter them out? Does the bisect algorithm provide a way to recover or re‑evaluate commits that were marked bad due to such transient errors?