Replace Fixed Sleeps with Karate's retry until for Eventually Consistent APIs

Stop guessing how long an eventually consistent endpoint needs

An order or job endpoint that reports PENDING for a few seconds before flipping to COMPLETED is a common source of test pain. The quick fix is a fixed sleep—wait five seconds, then assert once. That approach is slow when the backend finishes in 200 milliseconds and flaky when a slow CI runner pushes the transition past the sleep window. The test is guessing at a duration instead of waiting for a state change.

The better model is polling. Karate DSL lets you express that directly with configure retry and retry until. You set an explicit retry policy, issue an idempotent GET, and let the framework re-issue the request until the resource reaches the expected state. The final assertions stay strict, but the wait becomes bounded and observable.

The sleep anti-pattern

A fixed sleep couples your test to the worst-case latency of the system under test. If the job usually completes in one second but occasionally takes six, a five-second sleep fails intermittently. Raising the sleep to ten seconds makes the suite slower for every run, even the fast ones. Worse, a blind wait gives you no signal about whether the endpoint is progressing, stuck, or returning errors. When the test finally fails, the only diagnostic is a single response snapshot taken after the sleep expired.

Retry policy as code

Karate's retry mechanism turns the wait into configuration. configure retry accepts a JavaScript-like object with count and interval. The retry until keyword attaches a condition to the following request; Karate evaluates it after each response and re-issues the call while the condition remains false. This is polling, not a blind delay: the test stops as soon as the state transition happens.

Because the retry wraps the HTTP call, it works best with idempotent reads. A GET that fetches job status can be repeated safely. A POST that creates a resource usually cannot, unless the API documents an idempotency key.

Worked example: polling a job endpoint

Consider a feature that submits a job and then polls for completion. After the job is created, you need to wait for the backend to finish processing.

Feature: Poll an eventually consistent job endpoint

  Scenario: Wait for a job to reach COMPLETED
    * configure retry = { count: 20, interval: 2000 }
    Given url baseUrl + '/jobs/' + jobId
    And retry until response.status == 'COMPLETED'
    When method get
    Then status 200
    And match response.result == '#present'

Here baseUrl and jobId are variables defined earlier in the scenario or passed in from a calling feature. The retry policy allows up to 20 attempts spaced two seconds apart, for a maximum wait of roughly 40 seconds. If the job reaches COMPLETED on the third attempt, the scenario proceeds immediately. If the condition never becomes true, Karate exhausts the retries and fails the scenario. The generated report and log will show the request history so you can see whether the endpoint was returning PENDING, erroring, or slow to respond. Exact failure wording and report locations vary by Karate version, so confirm them against the version pinned in your build.

Run this from your normal Karate runner—JUnit 5, Maven, or Gradle. No special permissions are required beyond those needed to execute the suite. The main risk is total elapsed time: if your CI stage timeout is 30 seconds, a 40-second retry budget will kill the job before Karate can report a clean failure.

Guardrails and trade-offs

Retries are not free. They can mask a real outage by stretching a failure across 40 seconds instead of failing in two. They also lengthen feedback when something is genuinely broken. Treat the retry budget as a deliberate trade-off, not a default.

Aspect	Fixed sleep	retry until
Wait behavior	Blind, fixed duration	Polls until condition or budget exhausted
Fast-path latency	Always pays full sleep	Stops as soon as state changes
Failure signal	Single response after sleep	Full request history in report
Risk	Flaky or slow	Can hide outages; lengthens feedback

Keep the guardrails tight:

• Retry only idempotent reads. Repeating a POST, PATCH, or DELETE can duplicate side effects unless the API documents idempotency.
• Fail fast on 4xx. If the endpoint returns 400 or 404 while a resource is pending, retry until is the wrong tool. Distinguish “not ready yet” from “invalid request.”
• Bound the total wait. Multiply count by interval and keep it below your CI stage timeout.
• Isolate parallel scenarios. Use unique job IDs and avoid global setup so polling tests do not interfere with each other.
• Prefer push over pull. For long-running jobs, a webhook or a test-only status transition endpoint is often cheaper than polling.

Make the change deliberately

Start by inventorying fixed waits in your suite. Replace each one with an explicit configure retry block and a retry until condition on an idempotent GET. Log or report the retry count so you can see which endpoints are slow. Add a CI wall-clock budget per feature or tag so a polling test cannot silently become the slowest test in the build.

To verify the pattern locally, create a throwaway project pinned to your Karate version. Stub the job endpoint to return PENDING twice and then COMPLETED. Confirm the final match passes and measure the elapsed time. Then lower count so retries are exhausted and confirm the test fails clearly. Finally, run the same feature in the same parallel mode as CI to catch contention or shared-state issues before they reach the pipeline.