Scaling Data-Driven API Tests with Karate's Built-In Parallelism

The Problem: Parameterized Tests Without the Boilerplate

Most test frameworks treat data-driven testing as an afterthought. You annotate a test method with @ParameterizedTest, feed it a CSV source, and suddenly you're managing test runners, thread pools, and custom reporters. Karate flips this: every Scenario in a Feature file is a parallel execution unit by default when you use the JUnit 5 runner. No test-class scaffolding, no executor configuration — just write the data flow and Karate handles the concurrency.

Dynamic Scenario Generation at Runtime

The key is karate.forEach() or karate.call() inside a Background or Setup block. You read external data (CSV, JSON, even a database result set) and Karate materializes one Scenario per row at runtime. Each generated case gets its own report entry, timeline bar, and request/response log — no compilation step, no parameterized annotations.

Consider a CSV at src/test/resources/data/users.csv:

username,password,expectedStatus
alice,secret123,200
bob,wrongpass,401
charlie,secret123,200

A minimal Feature file that turns each row into a parallel login test:

Feature: Dynamic login validation

Background:
  * def users = read('classpath:data/users.csv')
  * karate.forEach(users, function(u){
      karate.call('login.feature', u)
    })

Scenario: placeholder (never runs)
  * print 'this scenario is replaced by generated cases'

The called login.feature receives each row as a JSON argument:

Feature: Login flow

Scenario: attempt login
  Given url baseUrl + '/auth/login'
  And request { username: '#(username)', password: '#(password)' }
  When method post
  Then status expectedStatus

Run with mvn test -Dkarate.env=dev and open target/karate-reports/karate-summary.html. You'll see three distinct Scenario entries, each with its own thread ID, duration, and embedded request/response payload.

Configuration That Travels With the Suite

Environment switching lives in karate-config.js (or karate-config-<env>.js). A single function returns a merged config object; the active profile is selected via -Dkarate.env=staging without touching test files. Example karate-config-staging.js:

function() {
  return {
    baseUrl: 'https://staging.api.example.com',
    tokenPath: '/oauth/token'
  };
}

Reports aggregate parallel runs automatically. The HTML report includes a timeline view, per-Scenario duration, and embedded logs — essential for debugging flaky tests that only surface under concurrency.

Trade-Offs You'll Hit at Scale

• Background re-execution: Heavy Background steps (token refresh, DB seed) run for every generated case. Use callonce in Background for expensive setup, but be aware it shares state across threads — mutate with care.
• IDE test discovery: IntelliJ and Eclipse won't list generated cases until execution because materialization happens at runtime. Debug by running the whole Feature file.
• Heap pressure: Datasets above ~10k rows can OOM during generation. Stream with karate.readAsStream() or split into multiple Feature files.
• Sequential generation: karate.forEach() executes in the generating thread; only the called Scenarios run in parallel. For true parallel data iteration, tag the called Feature with @parallel and use karate.call() with parallel: true (Karate 1.4+).

Verify It Works in Your Repo

1. Create a minimal Maven/Gradle project with karate-junit5 dependency.
2. Add the Feature files above and the CSV under src/test/resources/data/.
3. Run mvn test -Dkarate.env=dev.
4. Open target/karate-reports/karate-summary.html and confirm three Scenario entries with distinct thread IDs (add * print 'thread:', java.lang.Thread.currentThread().getId() in login.feature to see them).
5. Switch env: create karate-config-staging.js, run with -Dkarate.env=staging, verify requests hit the staging host in the report.

If the report shows parallel timelines and per-case logs, you've got data-driven parallelism without a single line of runner code.