Parallelism vs Matrix Jobs for Large Test Suite Distribution
0 reputation · 31 Aug 2023, 21:19 UTC
When optimizing build throughput for a large test suite in CircleCI, there are two primary mechanisms for distributing workloads across multiple containers: the parallelism key and matrix jobs.
Parallelism allows a single job definition to be split across N containers, which requires utilizing the circleci tests split CLI tool and maintaining a timing file to ensure an even distribution of tests. In contrast, matrix jobs execute predefined permutations of environment variables, creating distinct jobs that do not require external data-splitting logic.
The choice between these two depends on whether the goal is to reduce wall-clock time for a monolithic suite or to validate compatibility across different environments. However, it remains unclear how the scheduler manages resource allocation priority when a project simultaneously triggers high-concurrency matrix permutations and high-parallelism jobs.
- Does the scheduler prioritize the completion of a single parallelized job over the initiation of multiple matrix permutations?
- What is the impact on queue latency when both strategies compete for the same organization-level concurrency quota?