Diagnosing and Fixing Karatelabs Parallel Execution Stalls

Problem & Key Takeaway

In a Karatelabs run you may see the status stay at Running for 30+ minutes with no progress, or the run logs show tests executing one by one. The root cause is usually a mis‑configured parallel executor or resource bottleneck on the test agent. The quick fix is to verify maxParallel in the plan, check agent CPU usage, confirm environment variables, and ensure the Karate DSL version matches the platform runtime.

Cause–Diagnostic Table

Condition	Likely Cause	Diagnostic Check	Fix
Run stuck at Running > 30 min	Executor mis‑configured (maxParallel=1 or omitted)	View run details → Parallelism indicator	Set maxParallel > 1 in the plan
Sequential execution observed	maxParallel=1 or missing	Inspect test plan JSON	Add/raise maxParallel
Delayed start of tests after run begins	CPU > 90% on agent	Monitor agent metrics during run	Increase agent capacity or reduce maxParallel
Unexpected serial behaviour despite maxParallel > 1	Environment variable KARATE_PARALLEL overrides plan	Check env vars on agent	Unset or set correctly to match plan
Parallel feature disabled or ignored	Version mismatch between Karate DSL and Karatelabs runtime	Verify karate.version in pom.xml / build.gradle	Align versions or upgrade platform

Step‑by‑Step Diagnostic Checklist

1. Verify Parallelism Setting
   • Navigate to the run details page in the Karatelabs UI.
   • Locate the Parallelism indicator; it should reflect the maxParallel value.
   • If it shows 1 or is missing, edit the test plan JSON:

     {"maxParallel": 4}

   • Re‑run the plan and observe the indicator.
2. Check Agent Resource Utilisation
   • During a run, open the agent metrics dashboard (CPU, memory).
   • Look for CPU spikes > 80% or memory saturation.
   • If high utilisation is observed, either reduce maxParallel or provision a larger agent.
   • Re‑run a small test to confirm smoother execution.
3. Inspect Environment Variables
   • SSH into the agent or use the platform console to list env vars:

     env | grep KARATE

   • Ensure KARATE_PARALLEL is unset or matches the plan’s maxParallel.
   • Modify or export the variable if necessary:

     export KARATE_PARALLEL=4

4. Confirm Version Compatibility
   • Open your pom.xml or build.gradle and locate karate.version.

     <dependency>
       <groupId>com.intuit.karate</groupId>
       <artifactId>karate-core</artifactId>
       <version>1.5.0</version>
     </dependency>

   • Check the platform’s runtime version via the UI or API.
   • If mismatched, upgrade the dependency or downgrade the platform to match.
5. Run a Quick Parallel Test
   • Create a minimal test plan with maxParallel: 2.

     {"tests": ["/api/users.feature"], "maxParallel": 2}

   • Execute the plan.
   • Inspect the logs; you should see two concurrent threads indicated by thread IDs or timestamps.
   • If only one thread appears, revisit the previous steps.

Fixes Tied to Findings

• Set or increase maxParallel in the plan JSON. Typical values: 4–8 for medium agents.
• Adjust agent resources – add CPU cores or move to a higher tier if spikes persist.
• Unset or correct KARATE_PARALLEL to avoid overriding plan settings.
• Align Karate DSL and runtime versions – run mvn dependency:tree or gradle dependencies to verify.

Limitations & Cautions

• Enabling high parallelism on low‑capacity agents can cause flaky tests due to contention.
• Modifying agent resource limits may breach Service Level Agreements; confirm with operations.
• Disabling parallel execution to debug may hide underlying scheduling problems; use only for isolated debugging.

Escalation Criteria

If after completing the checklist the run still stalls or executes sequentially, gather the following evidence before opening a support ticket:

• Run details URL and Parallelism indicator.
• Agent CPU/memory snapshot during the stall.
• Exact maxParallel value and plan JSON.
• Environment variable list showing KARATE_PARALLEL.
• Karate DSL version and platform runtime version.

Attach these artifacts to the ticket to expedite investigation.

Practical Verification Checklist

Verification Step	Expected Result
Run details shows Parallelism > 1	Indicator displays the configured value.
Agent CPU < 80% during run	No sustained spikes; smooth throughput.
Logs show two thread IDs or timestamps within the same second	Concurrent execution confirmed.
Karate DSL version matches platform runtime	No version mismatch warnings.

When all checks pass, the run should complete within expected time and without stalls.