Diagnosing and Mitigating Flaky Tests with Mocha’s Retry Mechanism

Recognizable Condition

A test that sometimes passes and sometimes fails without any code changes is considered flaky. In Mocha output, you may see messages like Error: timeout of 2000ms exceeded or intermittent assertion errors such as Expected 5 to equal 6. This nondetermin behavior usually stems from timing issues or environmental factors rather than logic errors.

Cause/Diagnostic Table

Symptom	Likely Cause
Consistent timeout errors	Asynchronous operation not resolved (missing return, await, or done())
Intermittent assertion failures	Shared mutable state (globals, DOM, singletons) leaking between tests
Occasional network or service errors	External dependency latency or flakiness

Ordered Checks

1. Verify state cleanup - Ensure each test resets globals, clears DOM, or mocks in an afterEach hook.
2. Increase timeout temporarily - Add this.timeout(5000) inside the test or suite and see if the failure disappears.
3. Inspect async handling - Confirm that every asynchronous call is either returned, awaited, or signaled with done(). Look for missing return before promises.
4. Enable retries and observe count - Add this.retries(2) (or run with --retries 2) and run the suite; the reporter will show lines like retry 1 and retry 2 before a final pass.
5. Review external call logs - If the test hits a network endpoint, enable verbose logging or use a mocking library to see whether the call succeeds on retry attempts.

Fixes Tied to Findings

• Async not resolved - Return a promise, use await, or call done() after the async work.
• Shared state - Move state initialization into a beforeEach block and clean it in afterEach; avoid module-level variables.
• External flakiness - Stub the external service with libraries like sinon or nock, or introduce a configurable timeout and retry wrapper.
• Timing-dependent assertions - Replace hard-coded delays with polling or event-based checks (e.g., wait for a DOM element to appear).
• Residual nondeterminism - After confirming the test logic is sound, apply this.retries(n) with a small n (typically 1-2) to absorb occasional timing glitches.

Escalation Criteria

1. The test still fails after the configured retry limit (e.g., after this.retries(2) it fails on the third attempt).
2. Retries are needed in more than 20% of runs, indicating the underlying cause is not just occasional timing.
3. Increasing timeout or adding retries does not reduce failure frequency.

When any of the above occurs, consider redesigning the test: inject dependencies, use a fake server, or mark the test as skipped (this.skip()) until the root cause is fixed.

Verification Steps

To confirm that Mocha’s retry mechanism is working, create a minimal test file (e.g., flaky-demo.js) with the following content:

const assert = require('assert');

let attempt = 0;
describe('Flaky demo', function() {
  this.retries(2); // allow up to two retries

  it('passes on the second try', function() {
    attempt++;
    if (attempt < 2) {
      // fail the first attempt
      return Promise.reject(new Error('intentional failure'));
    }
    // succeed on retry
    assert.strictEqual(attempt, 2);
  });
});

Run the test from your project root:

mocha flaky-demo.js --retries 2

Expected behavior:

• The first run logs Flaky demo followed by 1) passes on the second try and a line indicating retry 1.
• The second run (first retry) logs retry 2 and then passes.
• If the test passes, the reporter shows a passing test with the retry count displayed.

To verify disabling retries works, repeat the run with this.retries(0) or mocha flaky-demo.js --retries 0 and confirm no retry lines appear and the test fails on the first attempt.

Limitations and Practical Check

• Retries do **not** reset timers or clear mocks; ensure mocks are restored in afterEach to avoid cross-test contamination.
• The this.retries() API is available starting with Mocha v5.0; older versions will silently ignore the call.
• Overusing retries can mask genuine bugs; apply them only after confirming the test logic is correct and the flakiness stems from timing or external factors.

Practical way to check the result: after each test run, inspect the terminal output for the strings retry 1, retry 2, etc. Their presence confirms Mocha executed the configured number of retry attempts.