Data‑Driven API Testing with Karate DSL: Using Scenario Outline and Examples

Problem: Repeating the Same API Call with Different Data

When you need to verify an endpoint against dozens of input variations—different user IDs, payloads, or query strings—writing a loop in JavaScript or duplicating scenarios quickly becomes hard to maintain. You want each iteration to appear as a distinct test in reports so failures are easy to trace, but you also want to keep the test definition in one place.

Thesis: Karate’s Scenario Outline + Examples Gives You Built‑In Data‑Driven Execution

Karate DSL treats a Scenario Outline as a template. Each row in the accompanying Examples table creates a separate Cucumber scenario, complete with its own name, pass/fail status, and reporting entry. No custom loops are required, and the feature works with all Karate versions ≥ 0.8.0 (empty‑cell handling changed in 0.9.6).

How the Outline Works

The outline defines placeholders using angle brackets, e.g. <userId>. When Karate parses the feature file, it substitutes each placeholder with the value from the corresponding column of the Examples table, generating one scenario per row.

Building the Examples Table

The table follows Cucumber’s pipe‑delimited syntax. Cells can contain primitives, JSON, XML, or even the result of the read function to load external files. Empty cells are treated as null starting with version 0.9.6; earlier versions turned them into empty strings, which could mask null‑check failures.

Worked Example: Testing a User Lookup Endpoint

Suppose you have a REST endpoint GET /users/{id} that should return a JSON user object. You want to verify the response for three different IDs, including one that does not exist.

Feature: User service data‑driven validation

  Scenario Outline: Retrieve a user by ID
    Given path 'users', 
    When method get
    Then status 
    And match response.id ==   # only for existing users

  Examples:
    | userId | expectedStatus |
    | 101    | 200            |
    | 102    | 200            |
    | 999    | 404            |

To run this:

1. Open a terminal in the project’s root directory (where pom.xml or build.gradle resides).
2. Ensure you have read/write access to the directory—no special privileges are needed beyond those required for a normal Maven/Gradle build.
3. Execute the test with Maven:

   mvn test -Dkarate.options="--tags @user"

   (Replace @user with the tag you add to the feature if you wish to filter; otherwise omit the option to run all scenarios.)
4. Or use the Karate CLI directly:

   karate -t @user src/test/java/feature/user-get.feature

What you should observe (based on Karate’s documented behavior):

• The console will list three separate scenario executions, each showing the substituted userId and expectedStatus values.
• The JUnit XML report will contain three distinct test cases, making it easy to identify which row failed.
• If you enable the HTML report (karate.options = '--tags @user --karate.summary'), the report will display three clearly labeled scenario blocks.

Trade‑Offs and Limitations

While Scenario Outline eliminates boilerplate, consider these practical points:

• Report size: With hundreds of rows the HTML report can become large and slow to open. Mitigation: split the data into multiple feature files or use the karate.summary flag to reduce detail.
• Empty‑cell handling: If you are on a version prior to 0.9.6, an empty cell becomes an empty string ("") rather than null. This can cause unexpected validation failures when you explicitly check for null fields. Upgrade to ≥ 0.9.6 or replace empty cells with the keyword null in the table.
• Parallel execution: Each generated scenario runs independently, so you can leverage Karate’s parallel runner (cucumber.parallel) without extra code.

Actionable Closing: Verify and Adopt

To confirm that the data‑driven approach works for your project:

1. Create a minimal feature file as shown above, adjusting the endpoint, path, and assertions to match your API.
2. Run the test with either Maven or the Karate CLI and check the console for three scenario lines.
3. Open the generated JUnit or HTML report and verify that each row appears as a separate test entry with the correct status.
4. If you notice overly large reports, apply the karate.summary flag or divide the Examples table into logical chunks.

By using Scenario Outline and Examples you keep your test logic in one place, gain clear per‑iteration reporting, and avoid writing manual loops—all with a feature that has been stable in Karate DSL for many releases.