Choosing Karma’s autoWatch vs singleRun for Development and CI

Decision

When you run tests with Karma you must decide whether to keep the process alive and watch for file changes (autoWatch) or to run once and exit (singleRun). The choice determines how quickly you see failures during development and whether the test run behaves predictably in continuous‑integration (CI) environments.

Constraints

• File‑watch support – If you rely on automatic re‑execution when source or test files change, autoWatch must be enabled.
• Graceful shutdown – In CI you need the process to terminate with an exit code that the pipeline can interpret; singleRun guarantees this.
• State leakage – Tests that depend on global state can pass in watch mode (state persists) but fail in singleRun; design tests to be idempotent.

Option Comparison

Mode	Feedback latency	Resource usage	CI suitability	Typical config
autoWatch	Instant – tests re‑run on every file change.	Higher – browser stays open, Node process stays alive.	Not ideal – process may hang, causing timeouts.	autoWatch: true, singleRun: false
singleRun	Delayed – tests run only once at startup.	Lower – process exits after completion.	Excellent – deterministic exit code, no hanging.	autoWatch: false, singleRun: true

Trade‑offs

• Speed vs Isolation – autoWatch gives you immediate feedback but can mask environment issues that only surface after a fresh process.
• Resource Consumption – Keeping the browser alive in watch mode consumes memory and CPU, which may be problematic on CI runners with limited resources.
• Flaky Tests – Global state that persists across runs can cause tests to pass in watch mode but fail in singleRun. This leads to false confidence.
• Pipeline Stability – A watch mode process that never exits will block the CI job, potentially causing a timeout or a manual kill.

Implementation

Most projects want autoWatch during local development and singleRun in CI. The following karma.conf.js toggles the mode based on the CI environment variable:

module.exports = function (config) {
  const isCI = process.env.CI === 'true';
  config.set({
    frameworks: ['jasmine'],
    files: ['src/**/*.js', 'test/**/*.spec.js'],
    browsers: ['ChromeHeadless'],
    singleRun: isCI,
    autoWatch: !isCI,
    // other common settings
  });
};

Run locally with:

npm test

Run in CI by setting the variable:

CI=true npm test

Validation

1. Local watch mode – Execute npm test and observe that Karma stays running. Edit a test file; the console should log a new “Running tests” message.
2. Single‑run mode – Execute CI=true npm test. Karma should run once and exit. Verify the exit code: echo $? should be 0 on success or a non‑zero value on failure.
3. Resource check – While autoWatch is active, monitor memory usage of the Node process and the headless browser; expect higher consumption than after a singleRun exit.

Limitations

• Some test runners (e.g., Karma with certain reporters) may not report a proper exit code unless singleRun is true.
• If you rely on autoWatch for debugging flaky tests, you may need to disable singleRun even in CI, but this defeats the deterministic pipeline goal.
• Global state must be cleaned up manually in the afterEach hook to avoid cross‑run contamination in singleRun mode.

By configuring Karma to switch modes automatically you get the best of both worlds: rapid feedback while developing and reliable, deterministic test runs in CI.