Choosing Between Manual and Automated Execution Strategies in qTest Manager

The Execution Dilemma: Human Intuition vs. Pipeline Speed

When structuring a release cycle in qTest, the primary technical decision is whether to execute tests manually via the Manager UI or automate the result reporting via API integrations. The wrong choice leads to either a bottleneck in the CI/CD pipeline (due to manual gates) or a lack of qualitative UX feedback (due to over-reliance on binary pass/fail scripts).

The goal is to determine which execution strategy fits specific test types while maintaining a single source of truth within a Test Cycle—the qTest entity that groups test executions for a specific release or sprint.

Comparison of Execution Strategies

Feature	Manual Execution	Automated Execution (API/Pulse)
Primary Use Case	Exploratory, UX, Ad-hoc	Regression, Smoke, API tests
Feedback Loop	Slow (Human-dependent)	Fast (Triggered by Build)
Maintenance	Low (Update Test Case steps)	High (Script & Framework updates)
Data Entry	Direct UI interaction	REST API / Plugin push
Scalability	Linear to headcount	Exponential via parallelization

Engineering Trade-offs

Manual Execution Flexibility

Manual execution allows a tester to deviate from the predefined steps to investigate a bug. In qTest, this is captured by adding a "Step" or an "Attachment" during the run. This is critical for User Acceptance Testing (UAT) where the "Pass" criteria are subjective.

Automated Execution Repeatability

Automated execution removes human variance. By pushing results from a framework (like Selenium or JUnit) into qTest via the API, you ensure that the exact same parameters are tested every time. However, this introduces a mapping risk: you must maintain a strict link between the External ID of the script and the qTest Test Case ID. If this mapping fails, the system may create duplicate test entities, cluttering the project hierarchy.

Implementation: Validating a Mixed-Mode Cycle

To achieve a balanced QA workflow, implement a mixed-mode Test Cycle. This allows leadership to see the overall progress of a release regardless of how the test was triggered.

Step 1: Manual Execution Validation

1. Navigate to the Test Execution tab in qTest Manager.
2. Create a new Test Cycle (e.g., "Sprint 24 Regression").
3. Add a Test Case to the cycle to create a Test Run.
4. Open the Test Run and manually change the status from Not Executed to Passed or Failed.
5. Verify the status transition is reflected in the Cycle's progress bar.

Step 2: Automated Result Push via API

For automated tests, results are pushed to qTest using the REST API. This requires an API Token and the specific runId of the test execution.

Command Execution: Run this from your CI/CD runner (e.g., Jenkins or GitHub Actions) with a service account possessing "Edit" permissions for the project.

curl -X PUT "https://{your-instance}.qtestnet.com/api/v3/projects/{projectId}/test-runs/{runId}" \
-H "Authorization: Bearer {api_token}" \
-H "Content-Type: application/json" \
-d '{"status": "passed", "comment": "Automated regression passed in Build #402"}'

Placeholders:

• {your-instance}: Your qTest cloud or on-prem URL.
• {projectId}: The unique ID of the qTest project.
• {runId}: The specific ID of the execution instance within the cycle.

Step 3: Verification of Results

To verify the integration, navigate to the Execution Progress report. A successful implementation will show a single report where some runs are marked as "Manual" (updated via UI) and others are updated via the API, providing a unified view of the release health.

Limitations and Risks

• Data Bloat: Automated pipelines that push results on every commit can generate thousands of execution logs. Implement a cleanup policy to archive old Test Cycles.
• Mapping Errors: If the API push targets the wrong runId, you may overwrite manual test results with automated ones. Always validate the External ID mapping before deploying a new automation suite.

Rollback Procedure

If an automated push incorrectly updates a batch of test runs, you must manually revert the status in the qTest Manager UI or execute a corrective API PUT request to reset the status to Not Executed for the affected runIds.