Diagnosing k6 Threshold Evaluation Failures: A Step-by-Step Guide

Recognizable Condition

Your k6 load test finishes with exit code 0 even though a performance target (e.g., 95th‑percentile latency under 500 ms) was clearly missed. Conversely, the test may fail with exit code 1 but the threshold summary shows all green. Both scenarios indicate that threshold evaluation did not behave as intended.

Cause and Diagnostic Table

Symptom	Likely Cause	Quick Check
Test exits 0 despite crossed threshold	Missing abortOnFail: true or --fail-on-threshold flag	Run k6 run --help | grep -A2 threshold and verify flag presence
Threshold reported as passed but metric value exceeds target	Metric selector mismatch (case‑sensitive tags, wrong built‑in name)	Export summary JSON and inspect thresholds object for evaluated metric names
Custom metric threshold never evaluates	Metric instantiated but zero samples recorded	Check --out json=metrics.json for metric entries
Checks threshold aggregates all checks, masking a single failure	Selector checks without tag filter	Add tag selector: checks{my_check:true}
Threshold works locally but differs in k6 Cloud	Cloud aggregates across load generators; local runs single instance	Compare --summary-export from both runs
Script throws no error but threshold ignored	JavaScript exception inside threshold expression (silent skip)	Wrap complex expressions in try/catch and log via console.error
Older k6 version ignores top‑level thresholds export	Using v0.47+ syntax on pre‑v0.47	Run k6 version and adjust to options.thresholds if needed

Ordered Diagnostic Checks

1. Verify threshold configuration is parsed. Add console.log(JSON.stringify(options.thresholds)) at the top of your script. Run k6 run script.js and confirm the printed JSON matches your intent.
2. Run a minimal failing test. Create a script with a deliberately impossible threshold:

   export const options = {
     thresholds: {
       'http_req_duration': ['p(99)<1']
     }
   };
   export default function () {
     http.get('https://httpbin.org/delay/1');
   }

   Run k6 run --fail-on-threshold script.js. Expect non‑zero exit. If exit is 0, the threshold engine is not evaluating.
3. Inspect evaluated thresholds. Run k6 run --summary-export=summary.json script.js. Open summary.json and examine the thresholds object: each entry shows pass, threshold, and value. This reveals whether the metric selector matched and what value was measured.
4. Validate metric names and tags. For custom metrics, run k6 run --out json=metrics.json script.js then filter with jq '.metrics | keys[]'. Ensure the metric name and tag keys exactly match the selector (case‑sensitive). Built‑in metrics use snake_case (e.g., http_req_failed, not http_req_failed_rate).
5. Check executor impact. If using externally-controlled executor, thresholds evaluate only after k6 pause/resume finalizes metrics. For ramping-vus or per-vu-iterations, evaluation occurs at test end.
6. Confirm CI exit‑code handling. CI must treat exit 0 as "thresholds passed OR not configured" and exit 1 as "threshold failed OR script error". Parse stdout for the thresholds: section to differentiate.

Fixes Tied to Findings

• Missing abort behavior: Add abortOnFail: true to each threshold definition or pass --fail-on-threshold on the CLI.

  export const options = {
    thresholds: {
      'http_req_duration{type:api}': [{ threshold: 'p(95)<500', abortOnFail: true }]
    }
  };

• Metric selector mismatch: Align selector with exact metric name and tags. For a custom Trend named api_latency tagged with endpoint:login, use 'api_latency{endpoint:login}': ['p(95)<300'].
• Zero‑sample custom metrics: Ensure the metric receives at least one sample. Add a fallback metric.add(0) if the code path may be skipped.
• Checks aggregation: Tag each check and reference the tag in the threshold: checks{login_success:true}': ['rate>0.99'].
• Silent JS exceptions: Wrap threshold expressions:

  thresholds: {
    'http_req_duration': [
      { threshold: (metric) => {
          try { return metric.values.p95 < 500; }
          catch (e) { console.error('Threshold eval error:', e); return false; }
        },
        abortOnFail: true
      }
    ]
  }

• Version mismatch: On k6 < v0.47, move thresholds under options.thresholds. On v0.47+, top‑level export const options = { thresholds: {...} } works. Do not mix styles.
• Cloud vs local discrepancy: Run the same script with K6_CLOUD_TOKEN=xxx k6 cloud script.js and compare summary.json from both. Adjust thresholds to account for aggregated percentiles across generators.

Escalation Criteria

Escalate to deeper investigation or k6 support when:

• Thresholds still do not evaluate after applying all fixes above.
• Memory spikes (OOM) occur during threshold evaluation — likely due to high‑cardinality tags (request IDs, timestamps). Mitigate with discardResponseBodies: true and limit tag values.
• CI pipeline reports inconsistent exit codes across runs with identical scripts — indicates flaky metric collection or executor timing issues.
• k6 Cloud shows threshold pass/fail that cannot be reproduced locally after exhaustive comparison of summary.json files.

Limitations and Practical Verification

Thresholds in k6 are evaluated at test end by default; real‑time abort requires abortOnFail or the xk6-threshold extension. Long‑running tests may waste resources before failure detection. Tag cardinality explosion can cause OOM; monitor memory with k6 run --out influxdb=... if needed.

Verification checklist:

1. Run k6 version to confirm version assumptions (v0.47+).
2. Execute the minimal failing script with --fail-on-threshold and verify non‑zero exit.
3. Produce summary.json and confirm the thresholds object contains expected pass/fail entries.
4. Cross‑check custom metric tags via --out json and jq.
5. If using Cloud, run both local and Cloud executions and diff the thresholds sections.

Following these steps will surface the root cause in the majority of threshold evaluation failures and provide a repeatable process for future test maintenance.