Question
Choosing a Chrome for Testing Revision Strategy in Puppeteer CI Pipelines
Lax Lane
0 reputation · 10 Jul 2021, 19:09 UTC
28K views1
Loading…
0 reputation · 10 Jul 2021, 19:09 UTC
In a CI environment, teams often face a trade‑off between reproducibility and staying current with Chrome security patches. Puppeteer releases pin a specific Chrome for Testing revision, yet the executablePath can be overridden to point to a system Chrome of a different major version. When Puppeteer v22 removed the legacy headless mode, the options headless: true (new headless) and headless: 'shell' (old chrome‑headless‑shell) introduce another layer of compatibility concerns, as the latter is sensitive to the exact Chrome revision.
The goal is to decide on a strategy that minimizes flaky test runs while ensuring the browser binary remains up‑to‑date with security releases. Constraints include the default cache path differences in Docker containers, the need to pin a revision for reproducibility, and the potential for CDP domain changes in newer Chrome builds.
Uncertainty remains about how often to bump the pinned Chrome for Testing revision and whether using headless: 'shell' offers a more stable fallback across Chrome upgrades.
Which approach—pinning a specific Chrome for Testing revision versus dynamically selecting the latest system Chrome—provides the best balance of stability and freshness in a CI pipeline?
How often should the pinned Chrome revision be updated to avoid security gaps without introducing new incompatibilities?
Does the choice of headless: true versus headless: 'shell' significantly affect test flakiness when the Chrome revision changes?
For CI pipelines, pinning a specific Chrome for Testing (CfT) revision provides the best balance of stability and freshness. Dynamically selecting the latest system Chrome introduces non-deterministic failures when the browser updates independently of the Puppeteer library, often leading to Chrome DevTools Protocol (CDP) mismatches.
To maintain security without sacrificing reproducibility, adopt a 4-to-6 week update cycle. This aligns with the Chrome stable release cadence, ensuring security patches are integrated while allowing teams to validate the new revision in a staging environment before updating the production pipeline.
To implement this in a Dockerized CI environment, avoid relying on system-installed Chrome. Instead, use the Puppeteer-managed download and cache the binary to prevent redundant network calls:
puppeteer# Example: Setting the cache directory for Docker persistence
export PUPPETEER_CACHE_DIR=/home/node/.cache/puppeteer
npm installThe choice between headless: true and headless: 'shell' significantly impacts flakiness during revision changes:
headless: true (New Headless): This is the recommended standard for Puppeteer v22+. It is a full browser implementation and is generally more resilient to revision updates.headless: 'shell' (Chrome-headless-shell): This is a stripped-down binary. While faster, it is highly sensitive to the exact Chrome revision and more prone to breaking when the underlying CDP implementation shifts.| Strategy | Reproducibility | Security Freshness | Risk Factor |
|---|---|---|---|
| Pinned CfT Revision | High | Controlled | Low (predictable) |
| Latest System Chrome | Low | Immediate | High (version drift) |
When bumping a pinned revision, perform these checks to ensure stability:
PUPPETEER_CACHE_DIR is being hit rather than a fresh download.headless: true renders the expected DOM elements by taking a few diagnostic screenshots during the first run of the new revision.Diagnostic Detail Required: Are you using puppeteer or puppeteer-core? If using puppeteer-core, you must manually manage the executablePath, which changes the recommendation for binary caching.
Use comments to ask for clarification. Post a solution as an answer.
No question comments on this page.