karate.callSingle Caching in Parallel Runs: Shared State and the JUnit 4 to JUnit 5 Boundary

Direct Answer

Yes, callSingle returns the exact same object reference to every scenario in the same JVM. The cache lives in a static ConcurrentMap keyed by feature path (see CallSingleCache in Karate source). No defensive copy is made on retrieval. If one scenario mutates the returned map or list, every other scenario sees that mutation — this is the root cause of order-dependent flakiness in parallel runs.

The Karate reference documentation states "run once, reuse result" but does not explicitly warn that the cached object is shared mutable state. Treat the returned value as effectively immutable by convention.

JUnit 4 vs JUnit 5 Runner Differences

JUnit 4 (karate-junit4)

• One Karate instance per test class.
• Parallelism achieved via Maven Surefire/Failsafe forkCount > 1 — each fork gets its own JVM and therefore its own callSingle cache.
• Setup runs once per fork, not once per entire build.

JUnit 5 (@Karate.Test with karate-junit5)

• Each annotated method receives a separate Karate instance.
• All methods execute in the same JVM (unless forkCount is also configured), so they share the same static cache.
• Enabling junit.jupiter.execution.parallel.enabled=true runs methods concurrently, increasing contention on the shared cached object.

Report Aggregation

• JUnit 4: one HTML report per fork; aggregation requires a post-build step.
• JUnit 5: per-method reports produced; set karate.junit5.report=true to let Karate's reporter aggregate into a single cucumber-html-report within that JVM. Cross-fork aggregation still needs a separate step (e.g., karate-report Maven plugin).

Mixing JUnit 4 and JUnit 5 in One Module

Discouraged. Surefire runs JUnit 4 tests; Failsafe (or Surefire with junit-platform-engine) runs JUnit 5 tests. They execute in separate JVM forks, so callSingle caches are not shared across the boundary — each fork re-runs its own setup.

Safe Consumption Pattern

Return immutable data (JSON, map of primitives) or a factory function from callSingle. If you must mutate, deep-copy in the scenario:

def setup = callSingle('classpath:auth/setup.feature')
def local = copy $setup  # Karate's built-in deep copy
local.newKey = 'value'   # safe mutation

Verification Steps (Run Once)

1. Confirm Karate version on classpath: mvn dependency:tree | grep karate (behavior matches CallSingleCache.java of that version).
2. Enable debug logging: -Dkarate.log.level=DEBUG — observe cache hits/misses and fork boundaries in console.
3. Run a minimal reproducer: two scenario outlines in parallel that both callSingle a feature returning a map, then each adds a distinct key. Inspect whether keys leak across scenarios.
4. Inspect target/karate-reports after a parallel run: multiple karate-summary.json files = separate forks; single file = single-JVM parallelism.
5. For JUnit 5, check Surefire/Failsafe config for forkCount and junit.jupiter.execution.parallel.mode to understand actual parallelism.

Assumptions & Uncertainty

• Assumes Karate 1.x (tested against 1.4.x source). Versions before 1.0 used a different cache key (scenario name + feature) — verify with Karate.version at runtime if on an older release.
• Cache scope is JVM/classloader; containerized test runners that reuse JVMs across builds (e.g., some Gradle daemon setups) may retain stale cache — not typical for Maven.
• No official mutation contract exists; the "effectively immutable" guidance is community convention, not a documented guarantee.

One diagnostic that changes the recommendation: What is the exact Karate version on your classpath (output of mvn dependency:tree | grep karate)? Pre-1.0 cache keys behave differently.