Modularizing GitLab CI/CD Pipelines Using the Include Keyword

Solving the Monolithic YAML Problem

As projects grow, the .gitlab-ci.yml file often becomes a "monolith"—a massive, difficult-to-maintain file where build, test, and deploy logic are tightly coupled. This makes it nearly impossible to share common patterns (like security scanning or deployment scripts) across multiple repositories without manual copy-pasting.

The include keyword solves this by allowing you to decompose your pipeline into smaller, reusable templates. The primary takeaway is that included files are merged into the main configuration, but the main .gitlab-ci.yml always retains the final word, allowing you to override global templates with project-specific settings.

Include Mechanisms and Configuration

Depending on where your templates live, GitLab provides three primary methods for inclusion. These are processed during the pipeline creation phase before any jobs are dispatched to runners.

1. Local Includes

Used for organizing a single project. You move specific stages (e.g., test-jobs.yml) into a subdirectory to keep the root clean.

2. Project Includes

Used for organizational standards. You store a "Golden Template" in a dedicated ci-templates project and reference it across the company.

3. Remote Includes

Used for external configurations hosted on a public or private HTTPS endpoint.

Worked Example: Parameterized Templates

Modern GitLab versions support inputs, which transform static YAML files into dynamic functions. This prevents you from having to hardcode environment names or image versions inside the template.

Template File: /templates/deploy-template.yml

spec:
  inputs:
    environment:
      default: staging
    cluster_id:
      default: "cluster-01"

deploy-job:
  stage: deploy
  script:
    - echo "Deploying to $[[ inputs.environment ]] on $[[ inputs.cluster_id ]]"

Main File: .gitlab-ci.yml

include:
  - local: '/templates/deploy-template.yml'
    inputs:
      environment: production
      cluster_id: "cluster-prod-05"

# Overriding a specific property of the included job
deploy-job:
  variables:
    DEPLOY_TIMEOUT: "600"

Implementation Risks and Limitations

While modularization increases flexibility, it introduces specific failure modes that can block pipeline execution.

Circular Dependencies

If file-a.yml includes file-b.yml, and file-b.yml includes file-a.yml, the GitLab YAML parser will fail during the validation phase. This prevents infinite loops but will stop your pipeline from starting entirely.

Include Depth and Complexity

Deeply nested includes (templates including templates including templates) can hit internal depth limits. This often manifests as a linting error. To avoid this, flatten your hierarchy; aim for a maximum of two levels of inclusion.

Security Risks of Remote Includes

Using remote includes pulls YAML from an external URL. If that external source is compromised, an attacker can inject malicious script blocks into your pipeline, which then execute with the permissions of your GitLab Runner. Recommendation: Use project includes for internal company standards to ensure the configuration is versioned and access-controlled within your own GitLab instance.

Verification and Diagnostics

Because the final pipeline is a merged result of multiple files, you cannot rely on looking at a single file to understand the pipeline's behavior.

The CI Lint Tool

To verify the merged configuration, navigate to CI/CD > Editor in your GitLab project. The Pipeline Editor provides a "View merged YAML" tab. This is the only way to see exactly how overrides and inputs have been resolved before committing the change.

Practical Check List

• Permissions: Ensure the user triggering the pipeline has Reporter access or higher to the project referenced in a project include.
• Pathing: Local paths are relative to the project root. Use a leading slash (e.g., /templates/file.yml) to ensure consistency.
• Job Names: If an included file and the main file both define a job with the same name, the main file's definition wins.

Rollback Procedure

If a template update breaks multiple pipelines, the fastest recovery is to pin the include to a specific git ref (branch, tag, or commit SHA) rather than using the default branch:

include:
  - project: 'org/ci-templates'
    ref: 'v1.2.0' # Pinning to a tag prevents breaking changes from 'main'
    file: '/deploy.yml'