Cypress version pinning: binary cache or Docker image for a repeatable test environment
0 reputation · 10 Mar 2021, 12:18 UTC
Our team runs Cypress end-to-end tests on developer laptops and in CI, and we want every run to use the same Cypress binary and browser versions. Cypress documents two routes: the official Docker images (cypress/included pins Cypress plus specific Chrome, Firefox, Edge and Electron versions) or pinning the npm package version and managing the binary cache ourselves.
The npm route worries us. The Cypress binary is downloaded into a cache (default ~/.cache/Cypress on Linux) rather than installed by npm, so package-lock.json alone does not seem to guarantee a hermetic install. Variables like CYPRESS_INSTALL_BINARY and CYPRESS_CACHE_FOLDER also appear to change whether installs stay reproducible, and we are unsure how they interact with a clean CI workspace. Assume a current Cypress major version; we will confirm exact tag names and defaults against the docs.
There is also a browser question: the bundled Electron browser is pinned by the Cypress version, but system Chrome floats unless the image pins it, and we are not sure how far Electron results can stand in for real Chrome.
1. Is a pinned package-lock entry plus a controlled cache sufficient for bit-for-bit repeatable installs, or is Docker the only reliable option?
2. Which cache-related environment variables must be fixed in CI to avoid silent drift?
3. How should we handle Chrome version skew if we stay outside Docker?