Stopping the 'Green Build' Lie: Using k6 Thresholds for Performance Gates

The Problem with 'Pass/Fail' in Performance Testing

Most CI/CD pipelines are great at catching functional regressions—if a test fails, the build stops. However, performance regressions often go unnoticed. A deployment might pass all functional tests while the p95 response time jumps from 200 ms to 800 ms. Manually reviewing load‑test graphs after every merge is not true continuous integration; it’s manual auditing.

The goal is to turn performance metrics into Service Level Indicators (SLIs) that can automatically fail a build. In k6, this is handled via Thresholds. Thresholds let you define a mathematical expression that must be true for the test to be considered a success. If the expression evaluates to false, k6 exits with a non‑zero code, signaling your CI runner (GitHub Actions, GitLab CI, etc.) to stop the pipeline.

Defining Performance Gates

Thresholds are defined inside the options object of your k6 script. You can target global metrics, such as the overall failure rate, or specific request groups using tags. This granularity is critical—an 500 ms response time might be acceptable for a PDF export but catastrophic for a health‑check endpoint.

Global vs. Targeted Thresholds

Global thresholds monitor the overall system health. For example, ensuring that 99 % of all requests succeed. Targeted thresholds use tags to isolate specific API endpoints, ensuring that a slow legacy endpoint doesn’t mask the performance of a new, optimized service.

Practical Example: Implementing a Performance Gate

Below is a minimal k6 script that demonstrates both global and targeted thresholds. It assumes k6 v0.40.0 or later.

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  stages: [
    { duration: '30s', target: 20 },
    { duration: '1m', target: 20 },
    { duration: '30s', target: 0 },
  ],
  thresholds: {
    // Global: 99% of all requests must be 2xx/3xx
    http_req_failed: ['rate<0.01'],
    // Global: 95% of all requests must be under 500 ms
    http_req_duration: ['p(95)<500'],
    // Targeted: The 'login' endpoint must be under 200 ms (p95)
    'http_req_duration{name:login}': ['p(95)<200'],
  },
};

export default function () {
  // Tagging the request allows the targeted threshold above to track this specific call
  http.get('https://test-api.k6.io/public/crocodiles/', { tags: { name: 'home' } });
  http.get('https://test-api.k6.io/public/crocodiles/1', { tags: { name: 'login' } });
  sleep(1);
}

Running and Verifying

Run the script from your terminal:

k6 run script.js

If the login endpoint takes 250 ms, k6 will print a red ✗ next to that threshold in the final summary and exit with a non‑zero status code. Verify the exit code immediately after the run with echo $? on Linux/macOS or echo $LASTEXITCODE on Windows.

Trade‑Offs and Limitations

• Flaky Builds: Performance never stays flat. Network jitter, noisy neighbors, and cold starts can cause transient spikes that trigger thresholds. Use realistic baselines and avoid overly tight limits on unstable environments.
• Averages vs. Percentiles: Relying on avg hides outliers. A 50 %/50 % split of 10 ms and 1 s latency averages to 505 ms, which looks “ok” but is unacceptable for half the users. Prefer percentiles (p95, p99).
• Resource Overhead: Calculating high‑percentile trends over millions of requests can increase memory usage on the load generator. Monitor the k6 process for spikes.
• Abort Early: Add abortOnThreshold: true to options to stop the test immediately when a threshold is breached, saving cloud compute time.

Verification Checklist

1. Force a Failure: Temporarily set a threshold to an impossible value (e.g., p(95)<1ms) and confirm the CI pipeline actually fails.
2. Tag Accuracy: Check the k6 summary to ensure the targeted metric (e.g., http_req_duration{name:login}) appears. A misspelled tag will silently ignore the threshold.
3. Baseline Stability: Run the test three times in a stable environment to establish realistic thresholds before committing them to source control.