Validating API correctness with Gatling checks and assertions

Problem: Load tests that only measure speed can miss broken APIs

When you run a Gatling simulation you often focus on response‑time charts and throughput numbers. If the service starts returning error payloads or missing fields, the test may still show “green” because the HTTP status is 200 and the timing looks fine. To catch functional regressions you need to validate the actual content of each response while the test is under load.

Thesis: Pair per‑request checks with global assertions

Gatling provides two complementary mechanisms:

• Checks – executed per virtual user, per request. They can assert status, headers, or body (JSONPath, regex, XPath, JMESPath) and optionally extract values for later requests.
• Assertions – evaluated once after the simulation finishes. They operate on aggregated metrics such as success rate, mean response time, or percentile thresholds and determine the overall pass/fail outcome of the test.

Using both lets you verify correctness on a per‑request basis while still enforcing service‑level objectives (SLOs) across the whole load.

Section 1: Writing a per‑request check

A check is attached to an http request with the .check method. The simplest form validates the HTTP status:

http("Get user")
  .get("/api/users/${userId}")
  .check(status.is(200))

If the status is not 200, the virtual user is marked as KO (knock‑out) and the failure appears in the console and HTML report.

You can also validate JSON content and simultaneously extract a field for reuse:

http("Create user")
  .post("/api/users")
  .body(StringBody("{ \"name\": \"Alice\" }")).asJson
  .check(
    status.is(201),
    jsonPath("$.id").saveAs("createdId")   // extracts the generated ID
  )

The saveAs step stores the extracted value in the session, making it available to subsequent requests via the ${createdId} placeholder.

Section 2: Chaining requests with extracted data

Once you have saved a value, you can reference it in later steps. This is essential for realistic scenarios where you create a resource then update or delete it:

val scn = scenario("User lifecycle")
  .exec(http("Create user")
    .post("/api/users")
    .body(StringBody("{ \"name\": \"Bob\" }")).asJson
    .check(status.is(201), jsonPath("$.id").saveAs("userId")))
  .exec(http("Get user")
    .get("/api/users/${userId}")
    .check(status.is(200), jsonPath("$.name").is("Bob")))
  .exec(http("Delete user")
    .delete("/api/users/${userId}")
    .check(status.is(204)))

If any check fails, the virtual user stops at that point (unless you mark the check as optional – see the limitation section). The failure is tied to the specific request, making debugging straightforward.

Section 3: Global assertions for SLOs

After the simulation ends, Gatling evaluates assertions defined with .assertions. Typical examples:

setUp(scn.injectConstantUsersPerSec(10) during (30.seconds))
  .assertions(
    global.responseTime.max.lt(500),          // 95th‑percentile < 500 ms
    global.successfulRequests.percent.gt(99) // > 99 % success
  )

These assertions do **not** stop the test early; they are evaluated once the workload finishes. If an assertion fails, the simulation exits with a non‑zero status, which CI systems can treat as a failed build.

Worked example: End‑to‑end validation

The following Scala simulation demonstrates the full flow:

1. Create a user and extract the generated ID.
2. Retrieve the user and verify the name.
3. Update the user (optional check).
4. Delete the user.
5. Assert that average response time stays under 300 ms and that at least 99.5 % of requests succeed.

import io.gatling.core.Predef._
import io.gatling.http.Predef._
import scala.concurrent.duration._

class UserApiSimulation extends Simulation {

  val httpConf = http.baseUrl("http://localhost:8080")

  val scn = scenario("Validate user API")
    .exec(http("Create user")
      .post("/api/users")
      .body(StringBody("{ \"name\": \"Charlie\" }")).asJson
      .check(status.is(201), jsonPath("$.id").saveAs("uid")))
    .exec(http("Get user")
      .get("/api/users/${uid}")
      .check(status.is(200), jsonPath("$.name").is("Charlie")))
    .exec(http("Update user")
      .put("/api/users/${uid}")
      .body(StringBody("{ \"name\": \"Charlie Updated\" }")).asJson
      .check(status.is(200)))
    .exec(http("Delete user")
      .delete("/api/users/${uid}")
      .check(status.is(204)))

  setUp(scn.injectConstantUsersPerSec(5) during (1.minute))
    .protocols(httpConf)
    .assertions(
      global.responseTime.mean.lt(300),
      global.successfulRequests.percent.gt(99.5)
    )
}

To run this simulation (assuming a Gradle project with the Gatling plugin):

# Run from the project root
./gradlew gatlingRun -Dgatling.simulationClass=com.example.UserApiSimulation

Required permissions: read access to the source directory and execute permission on the Gradle wrapper. No special privileges are needed beyond those required to start the injector JVM.

Trade‑offs and limitations

• CPU overhead – Complex JSONPath or regex checks add processing time on the injector. If you notice the injector CPU nearing 100 %, simplify the expressions or move heavy validation to a separate functional test suite.
• Assertions are post‑run – Because assertions are evaluated after the simulation, a failing threshold does not stop the test early. The full duration is still consumed, which can waste time in CI pipelines. Mitigate by keeping assertion thresholds realistic and using lightweight checks to catch obvious errors early.
• Module dependencies – JSONPath and JMESPath checks require the corresponding Gatling module (e.g., gatling-jsonpath) on the classpath. Forgetting to add it leads to a runtime error like NoSuchMethodError: jsonPath. Verify your build.gradle or pom.xml includes the module:

  io.gatling
  gatling-jsonpath
  3.11.0

To check that the module is present, run ./gradlew dependencies and look for gatling-jsonpath in the runtime classpath.

Practical way to verify the result

1. After the run, open the generated HTML report (target/gatling/<timestamp>/index.html). Look for the “KO” count; each KO corresponds to a check failure.
2. In the console, lines like REQUEST BUILD USER #1 - GET /api/users/123 - KO (status.is(200) failed) show the exact check that failed.
3. Examine target/gatling/<timestamp>/simulation.log for a chronological list of events; the “KO” flag appears next to the request that triggered it.
4. Finally, check the exit code of the Gradle command: a non‑zero exit indicates at least one assertion failed.

Actionable closing

Start by adding a basic status check to every request in your scenario. Then, for endpoints that return critical data, add a JSONPath check that also extracts identifiers for chaining. Keep global assertions focused on the SLOs that matter most to your business (e.g., 95th‑percentile latency < 250 ms, error rate < 0.5 %). Monitor injector CPU during a pilot run; if it spikes, simplify or sample the checks. With this approach your load tests will not only tell you how fast the system is, but also whether it is behaving correctly under load.