Bounding Jenkins Matrix Execution with Axes, Exclude, and Timeouts

Problem: matrix axes can exhaust executors

Jenkins Declarative Pipeline `matrix` expands one stage into a cell for every combination of axis values. Each axis is a named dimension, such as a JDK version or operating system, and each cell is one permutation that Jenkins schedules like an independent stage. A matrix with three axes of four values each creates 64 cells. Every cell needs one executor slot on the agent for the duration of its stage. When cells outnumber executors, the extra cells queue. When many cells run on one agent, they compete for CPU, memory, and disk I/O. The result can be stalled builds, out-of-memory kills, or a single hung cell holding an executor while the rest of the pipeline waits.

Useful takeaway

Bound the matrix before it runs. Keep the number of axes small, use `exclude` to remove invalid combinations, set `timeout` inside each cell so a hung test releases its executor, and isolate each cell's files with `dir` so concurrent cells do not overwrite each other. After a change, verify with Stage View labels, Build Executor Status, and per-cell variable output.

How matrix expansion works

An `axis` defines a name and a list of values. Jenkins multiplies the axis lists to produce cells. For example, axes `PLATFORM` with `linux`, `windows` and `JAVA_VERSION` with `11`, `17` produce four cells: linux-11, linux-17, windows-11, windows-17. An `exclude` block removes cells that match all listed axis values in that block, such as windows with JAVA_VERSION 11. Cells run concurrently up to the agent's configured executor count; they do not automatically get separate agents unless the pipeline or stage requests one.

Worked configuration

This Jenkinsfile runs unit and integration tests on Linux and Windows with JDK 11 and 17. It skips the automatic checkout in every cell, checks out once in a parent stage, excludes Windows with JDK 11, and applies a 15-minute timeout to each cell.

pipeline {
    agent any
    options {
        skipDefaultCheckout(true)
    }
    stages {
        stage('Checkout') {
            steps {
                checkout scm
            }
        }
        stage('Test Matrix') {
            matrix {
                axes {
                    axis {
                        name 'PLATFORM'
                        values 'linux', 'windows'
                    }
                    axis {
                        name 'JAVA_VERSION'
                        values '11', '17'
                    }
                    axis {
                        name 'TEST_TYPE'
                        values 'unit', 'integration'
                    }
                }
                exclude {
                    axis {
                        name 'PLATFORM'
                        values 'windows'
                    }
                    axis {
                        name 'JAVA_VERSION'
                        values '11'
                    }
                }
                exclude {
                    axis {
                        name 'PLATFORM'
                        values 'windows'
                    }
                    axis {
                        name 'TEST_TYPE'
                        values 'integration'
                    }
                }
                stages {
                    stage('Run Tests') {
                        steps {
                            echo "PLATFORM=${PLATFORM} JAVA_VERSION=${JAVA_VERSION} TEST_TYPE=${TEST_TYPE}"
                            timeout(time: 15, unit: 'MINUTES') {
                                dir("${PLATFORM}-${JAVA_VERSION}-${TEST_TYPE}") {
                                    sh './gradlew test'
                                }
                            }
                        }
                    }
                }
            }
        }
        stage('Teardown') {
            steps {
                cleanWs()
            }
        }
    }
    post {
        always {
            cleanWs()
        }
    }
}

Where to run: save this as `Jenkinsfile` in the repository and configure a Pipeline job in Jenkins to build from SCM. The user creating or editing the job needs `Job/Configure` permission; running the build needs `Job/Build`. The `checkout scm` step requires SCM credentials configured in Jenkins.

What each part does:

• `skipDefaultCheckout(true)` prevents Jenkins from cloning the repository once per cell, which would duplicate network and disk load.
• The outer `Checkout` stage runs `checkout scm` once so all cells share the same source tree.
• Two `exclude` blocks remove three of the eight possible cells: Windows with JDK 11, and Windows with integration tests. Five cells remain.
• `timeout(time: 15, unit: 'MINUTES')` wraps the cell body. If `./gradlew test` hangs, only that cell aborts after 15 minutes; other cells continue.
• `dir(...)` gives each cell a unique subdirectory so parallel cells do not collide on build outputs.
• `cleanWs()` in `Teardown` and in the top-level `post { always { ... } }` removes the workspace after the build, even if the pipeline is aborted.

Limits and common mistakes

• Combinatorial explosion: five axes with three values each produce 243 cells. That many cells can exhaust executors and queue for hours. Limit axis cardinality or prune aggressively with `exclude`.
• Shared workspace collisions: without `dir`, concurrent cells write to the same workspace and overwrite each other's artifacts.
• Missing per-cell timeout: a hung cell blocks an executor indefinitely. Wrap cell steps in `timeout`, not just the whole pipeline.
• Stash size limits: `stash` and `unstash` move files through the controller. Files larger than about 100 MB degrade controller performance; use external artifact storage such as S3 or Artifactory for large outputs.
• No cell-level checkpointing: if the Jenkins controller restarts, the matrix starts from the first cell. Design test steps to be idempotent so a restart does not leave partial state.
• Executor assumption: cells share the agent where the `Test Matrix` stage runs. They do not fan out to separate agents automatically. For per-cell isolation, use a Kubernetes agent at the pipeline level so each cell runs in its own pod, subject to cluster capacity.

Diagnostic decision

If Stage View shows many cells waiting, reduce axis values or add `exclude`. If Build Executor Status shows all agent slots occupied while new builds queue, add per-cell timeouts or switch to Kubernetes agents so cells do not contend on a single static agent. If `echo` prints empty variable names, the Declarative Pipeline plugin version may not inject axis variables as environment variables; check the plugin version and syntax.

Verification steps

1. Commit the Jenkinsfile to a feature branch or create a new Pipeline job in Jenkins pointing at it.
2. Run the build and open Stage View. Each cell should appear as a separate node labeled with axis values, for example `Run Tests (PLATFORM=linux, JAVA_VERSION=11, TEST_TYPE=unit)`.
3. Confirm that excluded combinations, such as Windows with JDK 11, do not appear in Stage View.
4. Open Build Executor Status during the build. The number of active executors on the agent should not exceed the agent's configured executor count.
5. Check the console log for the `echo` line to confirm axis variables are injected per cell.
6. To test timeout behavior, temporarily add `sleep 1000` inside the `dir` block of one cell while keeping the 15-minute timeout. Confirm only that cell aborts after the timeout while other cells continue.
7. After the build, confirm the workspace is empty or removed, indicating `cleanWs()` ran.

Rollback

This change modifies the Jenkinsfile in source control. If the matrix causes excessive queueing, revert the commit or temporarily replace the `matrix` stage with a single sequential stage, then push and trigger a new build. Workspace and build records from the previous run remain unless manually deleted.

Practical check

The practical check is Stage View plus Build Executor Status: labels should match the expected non-excluded combinations, active executors should stay within agent capacity, and each cell should finish or abort independently within its timeout.

Version assumptions: Jenkins 2.400 or later with Declarative Pipeline plugin 2.2100 or later.