How to deploy and configure BrowserStack for automated cross‑browser testing in a CI pipeline?
0 reputation · 27 May 2024, 08:37 UTC
0 reputation · 27 May 2024, 08:37 UTC
Integrating BrowserStack into a CI/CD pipeline allows tests to run on real browsers and devices without local infrastructure. The goal is to trigger automated cross‑browser runs from the pipeline, capture results, and handle failures. Constraints include securely storing credentials, configuring desired capabilities for each test, and ensuring the test framework can communicate with BrowserStack’s remote WebDriver endpoint. The pipeline must also be able to handle asynchronous job provisioning and collect logs without manual intervention. Other uncertainties involve how to manage large numbers of browser/device combinations and how to correlate test failures with specific BrowserStack job details.
What is the recommended approach to securely store and retrieve BrowserStack credentials in the CI environment? How can desired capabilities be specified so that the test framework can target the correct browser/device combinations? What mechanisms can be used to collect and correlate BrowserStack job logs with pipeline failures?
26525 reputation · 27 May 2024, 17:34 UTC
In most CI systems you can store secrets as encrypted environment variables. For example, in GitHub Actions add BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESS_KEY to the Secrets section. In Jenkins, use the Credentials store and reference them with ${{ credentials('browserstack-cred') }}. The test code should read these values from the environment:
const bsUser = process.env.BROWSERSTACK_USERNAME;
const bsKey = process.env.BROWSERSTACK_ACCESS_KEY;
Never hard‑code credentials in source or commit them to a repository.
Capabilities are a JSON map that tells BrowserStack which browser, OS, and device to launch. Keep a single capabilities.json file and merge it with test‑specific overrides. Example for Chrome on Windows 10:
{
"browser": "chrome",
"browser_version": "latest",
"os": "Windows",
"os_version": "10",
"browserstack.debug": true,
"browserstack.networkLogs": true
}
For mobile testing add:
{
"device": "iPhone 14 Pro",
"realMobile": true,
"os_version": "16"
}
In the test framework (e.g., WebDriverIO, Selenium, Playwright) pass the capabilities when creating the remote driver:
const driver = await remote({
protocol: 'https',
hostname: 'hub-cloud.browserstack.com',
port: 443,
path: '/wd/hub',
user: bsUser,
key: bsKey,
capabilities: caps
});
Configure your CI job to run the test suite. If the job is long‑running or you need to run many combinations, parallelize using the CI’s matrix feature:
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
caps: [
{"browser":"chrome","os":"Windows","os_version":"10"},
{"browser":"firefox","os":"macOS","os_version":"14"}
]
steps:
- uses: actions/checkout@v3
- name: Install deps
run: npm ci
- name: Run tests
env:
BROWSERSTACK_USERNAME: ${{ secrets.BROWSERSTACK_USERNAME }}
BROWSERSTACK_ACCESS_KEY: ${{ secrets.BROWSERSTACK_ACCESS_KEY }}
run: npm run test -- --capabilities=${{ toJson(matrix.caps) }}
BrowserStack may take a few seconds to spin up a session. The WebDriver client automatically waits for the session to be ready, but you can add a retry loop if you see timeouts. Use a short implicitWait and catch SessionNotCreatedException to retry.
After a test finishes, BrowserStack exposes logs via the browserstack-results-api and a browserstack.local identifier if you used Local Testing. The simplest approach is to use the browserstack-status property in the test result object and then call the REST API:
const jobId = driver.sessionId;
const url = `https://api.browserstack.com/automate/sessions/${jobId}.json`;
const auth = Buffer.from(`${bsUser}:${bsKey}`).toString('base64');
const logs = await fetch(url, { headers: { Authorization: `Basic ${auth}` } });
const data = await logs.json();
console.log(data.session.status);
Include the jobId in your CI failure message so a developer can click the link in BrowserStack’s dashboard and view the detailed console, video, and network logs.
BrowserStack automatically cleans up sessions, but if you start a Local Testing tunnel you must close it in a finally block or CI cleanup step.
afterAll(() => {
if (local) local.stop();
});
Use comments to ask for clarification. Post a solution as an answer.
26,525 reputation · 27 May 2024, 16:15 UTC
One thing the existing answer doesn't cover: a session that finishes on BrowserStack isn't automatically marked passed or failed. The hub only knows the browser closed, so unless your framework explicitly reports the outcome, the Automate dashboard shows sessions with no meaningful result — which defeats the goal of correlating CI failures with BrowserStack jobs.
With Selenium you send a small script after each test, e.g.:
driver.execute_script(
'browserstack_executor: {"action": "setSessionStatus", '
'"arguments": {"status":"failed","reason":"assertion mismatch"}}');
Most official SDKs (WebDriverIO, Playwright, Cypress plugins) wrap this for you, but it has to be enabled or called in a teardown hook.
Second, set buildName to something unique per pipeline run (e.g. the CI build ID) and a stable projectName. That groups every parallel session from one run together in the dashboard, so a failed matrix job maps directly to a filterable build. Without it, hundreds of sessions from different runs interleave and debugging becomes guesswork.
Two caveats: capability key names differ between legacy JSONWire style and W3C bstack:options, so verify against the current docs for your framework version; and confirm your plan's parallel limit before sizing the CI matrix, since excess sessions queue rather than fail, which can look like a hang.