Choosing a qTest Execution Method: Manual, Automated, or Hybrid

Decision Context

When you start a new test suite in qTest you must decide how the tests will be executed: manual through the UI, automated via CI/CD pipelines, or a hybrid mix. The choice affects effort, traceability, and integration complexity.

Constraints to Consider

• Team skill set – Do you have experienced automation engineers?
• Test coverage – Are there exploratory or high‑risk areas that benefit from manual control?
• CI/CD maturity – Is your pipeline ready to publish results to qTest?
• Data integrity – Do you have a reliable mapping between local test identifiers and qTest IDs?
• Security – Does the API token have the ‘Test Execution’ scope?

Option Comparison

Method	Effort	Traceability	Automation Readiness	Common Risks
Manual	High – testers log each run, update results, and attach evidence.	Excellent – every step is visible in the UI.	None – no integration required.	Human error, inconsistent data, slow regression.
Automated	Low – once the pipeline is wired, runs are reproducible.	Good – results are stored in qTest, but you lose step‑by‑step detail unless the framework captures it.	High – requires stable API, correct ID mapping, and idempotent uploads.	Duplicate or orphaned results, authentication failures.
Hybrid	Medium – manual for exploratory, automated for regression.	Strong – you maintain manual visibility where needed.	Medium – need both UI work and API integration.	Complex coordination, risk of inconsistent data.

Trade‑Offs Explained

• Control vs. Speed – Manual gives full control but slows down regression cycles. Automation speeds up but may hide subtle manual steps.
• Traceability vs. Maintenance – Manual logs every action, while automation requires maintaining scripts and ID mappings.
• Risk of Data Duplication – Automated uploads without proper run ID handling can overwrite previous runs.
• Security – The API token must have the correct scope; otherwise authentication will fail and you’ll see 401 responses.

Concrete Implementation Example

Below is a minimal example of creating a test run and posting results via qTest’s REST API (v1). Replace placeholders with your environment values.

1. Create a Test Run (UI or API)

Assuming you already have a run ID, you can skip this step. If you need to create one via API:

curl -X POST "https://{qtest_domain}/api/v1/runs" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {api_token}" \
  -d '{
  "projectId": {project_id},
  "name": "Regression Run 2026‑09‑17",
  "description": "Automated Selenium regression",
  "runDate": "2026-09-17T00:00:00Z",
  "runType": "Regression"
}'

Check the response JSON for a runId field. If the request fails, verify that {api_token} has the Test Execution scope.

2. Prepare Result Payload

qTest expects results in a specific JSON format. A minimal payload for two tests looks like:

{
  "testResults": [
    {
      "testId": 12345,
      "status": "pass",
      "comment": "All assertions met"
    },
    {
      "testId": 67890,
      "status": "fail",
      "comment": "Element not found"
    }
  ]
}

3. Upload Results

curl -X POST "https://{qtest_domain}/api/v1/runs/{runId}/results" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer {api_token}" \
  -d @results.json

Expected response: HTTP 200 with a summary of processed results. A non‑200 status indicates an issue such as an invalid runId or insufficient permissions.

4. Verify in the UI

• Navigate to the Test Run page.
• Confirm that the status and comment fields match the payload.
• Check the run metrics (total, passed, failed) to ensure they reflect the uploaded data.

Verification Checklist

1. Run the API call to create a run and capture the runId.
2. Execute the automated test suite and generate the JSON payload.
3. POST the payload to the /results endpoint.
4. Open the run in qTest UI and confirm data integrity.

Limitations & Practical Checks

• The example assumes qTest v1 API. Future versions may change endpoints.
• Duplicate uploads without unique runId can overwrite data; use a new run for each pipeline execution.
• Ensure your CI environment can reach the qTest domain and that firewalls allow outbound HTTPS.
• Test the entire flow in a sandbox project before deploying to production.

Summary

Choose manual execution when you need granular control and traceability, automation when speed and repeatability are priorities, and hybrid when you want the best of both worlds. The REST API allows automated pipelines to push results directly into qTest, but careful handling of run IDs, test IDs, and permissions is essential to avoid data corruption.