Enforcing Trust Boundaries in Gatling with Response Checks

The Problem: Silent Failures in Load Tests

A common failure mode in performance testing is the "successful" test that produces invalid results. When a load generator sends thousands of requests, a server might begin returning 500 Internal Server Errors or empty JSON payloads. If the test only measures response time and HTTP status codes, it may miss critical business-logic failures—such as a missing authentication token or a corrupted data field—that render the rest of the user scenario useless.

The takeaway is that HTTP response checks must be used as trust boundaries. By validating the contract of every request, you ensure that a virtual user only proceeds if the system under test (SUT) has provided a valid state, preventing "ghost" traffic that skews performance metrics.

The Smallest Suitable Design

To implement a trust boundary, you integrate checks directly into the http request chain. In Gatling (Scala DSL), a check is a validation performed on the response before the virtual user moves to the next step.

The most efficient design uses a layered validation approach: first validate the transport layer (status code), then the structure (headers), and finally the payload (JSON path).

Implementation Example

The following configuration demonstrates a login sequence where the second request depends on a valid token from the first. This is run within a Gatling Simulation class.

import io.gatling.core.Predef._
import io.gatling.http.Predef._

val scn = scenario("Trust Boundary Example")
  .exec(http("Login Request")
    .post("/api/login")
    .body(StringBody("{\"user\":\"test\",\"pass\":\"secret\"}"))
    // Trust Boundary 1: Transport and Payload
    .check(status.is(200))
    .check(jsonPath("$.token").saveAs("authToken"))
  )
  .pause(1)
  .exec(http("Secure Data Request")
    .get("/api/data")
    .header("Authorization", "Bearer ${authToken}")
    // Trust Boundary 2: Business Logic Validity
    .check(status.is(200))
    .check(jsonPath("$.status").is("success"))
  )

Trust and Data Boundaries

In this architecture, the .saveAs("authToken") method creates a data boundary. If the jsonPath("$.token") check fails, the variable authToken is not populated correctly.

By default, Gatling marks the request as KO (failed) but continues the scenario. To strictly enforce the trust boundary and stop the virtual user immediately upon a contract violation, you must use the exitHereIfFailed modifier. This prevents the generator from sending subsequent requests with null or invalid tokens, which would otherwise flood your logs with 401 Unauthorized errors and artificially inflate your error rate.

Operational Checks and Performance Impact

While checks are powerful, they consume CPU cycles on the load generator. To maintain measurement accuracy, follow these operational constraints:

• Avoid Regex on Large Payloads: Use jsonPath or css selectors. Regular expressions on multi-megabyte responses can cause CPU spikes on the generator, leading to "coordinated omission" where the generator cannot keep up with the SUT.
• Limit Deep Nesting: Complex JSON paths that require scanning large arrays increase latency in the check phase.
• Sampled Validation: If you have 10,000 concurrent users, you do not need to validate every single byte of every response. Use doIf blocks or a random feeder to perform deep payload checks on only 1% of requests.

Failure Modes and Design Shifts

Failure Mode	Symptom	Design Adjustment
Brittle Contract	Tests fail due to minor UI/JSON key changes, not performance.	Shift from is("exact value") to exists or substring.
Generator Bottleneck	High CPU on load injector; response times appear higher than they are.	Remove non-essential checks; move validation to a separate functional test suite.
Intermittent State	Flaky failures due to dynamic timestamps in responses.	Use custom Scala functions for range-based validation instead of exact matches.

Verifying the Boundary

To verify that your trust boundaries are functioning, perform these three checks:

1. Intentional Mismatch: Change a jsonPath expectation to a non-existent field. The Gatling report should show the request as KO.
2. Dependency Break: Force the first request (Login) to fail. Verify that the second request (Secure Data) is either not executed (if using exitHereIfFailed) or fails with a 401, confirming the data boundary is linked.
3. Report Analysis: Check the "Errors" tab in the generated HTML report. It should explicitly list the failed check (e.g., jsonPath("$.token") failed) rather than a generic Connection Refused.