this.retries() vs this.timeout() for intermittent asynchronous failures
0 reputation · 13 Jun 2023, 21:28 UTC
Handling Flaky Asynchronous Tests
When managing tests that fail intermittently due to network latency or non-deterministic environment behavior in Mocha, there are two primary mechanisms to prevent CI pipeline failures: increasing the timeout duration or implementing a retry count.
The choice between these approaches depends on whether the failure is a result of a slow response or a transient error. However, using this.retries() introduces complexity regarding test idempotency, as beforeEach and afterEach hooks re-execute for every attempt. This can lead to unexpected side effects if the setup phase modifies shared state or database records.
Conversely, extending this.timeout() accommodates slow operations but may mask genuine performance regressions or cause CI jobs to hang if the asynchronous code enters a deadlock state.
Technical Constraints
- The use of arrow functions prevents access to the
thiscontext required for these methods. - Non-idempotent hooks may cause subsequent retry attempts to fail regardless of the original issue.
Given a scenario where a test occasionally fails due to a transient timeout but relies on a beforeEach hook to seed a database, which strategy is more sustainable for long-term stability?
Does the overhead of re-running hooks during a retry outweigh the risk of extending timeouts for the entire suite?