Stopping Flaky CI: Using k6 Thresholds for API SLA Validation

The Problem: Load Tests That Only Tell You 'It Finished'

Many teams treat load tests as a visual exercise: they run a script, look at a graph, and decide if the system looks "healthy." The problem is that human interpretation is subjective and doesn't scale. When a load test is integrated into a CI/CD pipeline, a script that completes without crashing is not the same as a script that proves your API meets its Service Level Agreement (SLA).

Without automated pass/fail criteria, you risk deploying a version of your service that technically "works" but has degraded latency from 200ms to 800ms—a regression that only becomes visible once users start complaining. The solution is to move from observation to validation using k6 thresholds.

Defining Your Success Criteria

Thresholds in k6 are essentially assertions for your performance metrics. Instead of manually checking the p95 (95th percentile) response time after a run, you define that value as a hard requirement. If the metric exceeds the threshold, k6 exits with a non-zero code, failing the CI build automatically.

There are two primary types of thresholds to consider:

• Global Thresholds: These apply to the entire test run across all Virtual Users (VUs) and iterations. These are best for overall system health.
• Per-VU Thresholds: These ensure that no single user experience is catastrophically bad, even if the average looks fine.

Practical Implementation: SLA Validation

To implement thresholds, you define a thresholds object within your k6 options. You can target built-in metrics like http_req_duration (the time between sending a request and receiving the response) or http_req_failed (the percentage of failed requests).

Run the following script on your local machine or CI runner using k6 run script.js. Ensure you have k6 installed (v0.40.0 or later is assumed for standard threshold syntax). No special permissions are required beyond network access to the target API.

import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
  vus: 10,
  duration: '30s',
  thresholds: {
    // The p95 response time must be below 500ms
    http_req_duration: ['p(95)<500'],
    // The error rate must be less than 1%
    http_req_failed: ['rate<0.01'],
    // 99% of all checks must pass
    'checks': ['rate>0.99'],
  },
};

export default function () {
  const res = http.get('https://test-api.k6.io/public/crocodiles/');

  check(res, {
    'status is 200': (r) => r.status === 200,
  });

  sleep(1);
}

How to Read the Results

When the test finishes, k6 provides a summary in the terminal. Look for the thresholds section. A green checkmark indicates the SLA was met; a red cross indicates a failure. If http_req_duration shows p(95)=620ms, the test will fail because it exceeded the 500ms limit, and k6 will exit with a non-zero status code that your CI runner interprets as a failed job.

The Danger of Arbitrary Numbers

The most common mistake in performance engineering is picking "round numbers" (like 1 second) as thresholds without a baseline. If your current production p95 is actually 1.2 seconds, setting a threshold of 1 second will cause your CI pipeline to fail immediately, leading the team to ignore the alerts as "noise."

The Baseline Workflow:

1. Run a baseline test on a stable environment with a known load.
2. Analyze the resulting p95 and p99 metrics from the summary output.
3. Set your threshold slightly above the baseline to allow for minor environmental jitter, or set it strictly to your business SLA if that SLA is already validated as achievable.

Limitations and Trade-offs

While thresholds are powerful, they have limitations depending on your execution environment. When running distributed tests (such as via k6 Cloud), thresholds are aggregated across all load zones. This means a massive latency spike in one geographic region might be "smoothed out" by fast responses in another, potentially masking a regional outage.

To verify the result of a threshold failure, always check the http_req_duration distribution in the final summary to see if the failure was caused by a few extreme outliers or a general shift in performance. You can also export results with k6 run --out json=results.json script.js and compare percentile values across runs to confirm your thresholds remain reasonable over time.

Closing Action

Stop treating load tests as documentation and start treating them as tests. Identify your most critical API endpoint, run a baseline to find its current p95 latency, and add a thresholds block to your k6 script today. This transforms your performance testing from a manual review process into an automated quality gate.