Building and Consuming Custom CircleCI Orbs: A Practical Guide for Engineering Teams

The Problem: Repeating Configuration Across Projects

When a team starts multiple repositories, each pipeline often ends up with the same set of jobs: lint, test, build, deploy. Even a small change—adding a new environment variable or updating a Docker image—requires editing every config.yml file. That duplication inflates maintenance costs and introduces drift between projects.

Thesis: Orbs as a Modular, Versioned Solution

CircleCI Orbs pack jobs, commands, and executors into a single, declarative package. Once published, any project can reference the orb by name and version, automatically pulling in the exact code it needs. This reduces duplication, enforces consistency, and speeds onboarding.

Key Benefits

• DRY Pipelines – One source of truth for shared logic.
• Consistent Behavior – Declarative definitions prevent accidental changes.
• Version Pinning – Lock a project to a known stable orb release.
• Semantic Versioning – Increment patch for bug fixes, minor for new features, major for breaking changes.

Step 1: Create a Custom Orb

Start by creating a directory for your orb. The orb root contains orb.yml, which defines jobs, commands, and executors.

my-orb/
├── orb.yml
└── src/
    └── build.sh

Example orb.yml that defines a simple build job:

version: 2.1

executors:
  node:
    docker:
      - image: cimg/node:18.12.0

commands:
  install-deps:
    description: "Install npm dependencies"
    steps:
      - run: npm ci

jobs:
  build:
    description: "Run unit tests and build"
    executor: node
    steps:
      - checkout
      - install-deps
      - run: npm test
      - run: npm run build

Notice the declarative syntax: no scripts are hidden; everything is visible in the orb file. This transparency aids code review and debugging.

Step 2: Pack and Publish the Orb

To publish, you need a CircleCI account with an API token and write access to an orb namespace (e.g., my-org/my-orb). Run these commands from the orb root:

1. Validate the configuration locally – Catch syntax errors before publishing.

   circleci config validate --orb orb.yml
   

   Requires the circleci CLI installed. No permissions beyond local execution.

2. Pack the orb – Create a distributable package.

   circleci orb pack . -o my-orb-0.1.0.yml
   

   Replace 0.1.0 with the intended semantic version.

3. Publish to the registry – Push the orb to CircleCI.

   circleci orb publish my-orb-0.1.0.yml my-org/my-orb@0.1.0
   

   You will be prompted for your API token if not already configured.

After publishing, the orb appears in the CircleCI Orb Registry, where you can view documentation and changelogs.

Step 3: Consume the Orb in a Project

In any repository, reference the orb in .circleci/config.yml:

version: 2.1
orbs:
  my-orb: my-org/my-orb@0.1.0

jobs:
  test:
    docker:
      - image: cimg/node:18.12.0
    steps:
      - checkout
      - my-orb/build

workflows:
  test-and-build:
    jobs:
      - test

Key points:

• Version Pinning – The @0.1.0 lock ensures the project runs the exact orb release.
• Command Invocation – my-orb/build calls the orb’s job directly.
• Local Validation – Run circleci config validate before pushing to catch missing orb references.

Testing and Verification

After consuming the orb, run a pipeline and inspect the logs:

1. Trigger a build – Push the change to GitHub and watch CircleCI execute the test job.
2. Inspect job logs – Verify that npm ci, npm test, and npm run build ran as expected.
3. Check orb version – In the job log, the orb name and version appear in the “Job” header.

For automated verification, add a test job that runs circleci orb pack and circleci orb publish --dry-run to confirm the orb’s integrity before merging.

Trade‑offs and Limitations

• Development Overhead – Each orb version requires a CI run to validate the orb’s behavior. This can slow down a rapid iteration cycle.
• Debugging Complexity – Bugs inside an orb affect all consuming projects. Thorough testing and version pinning mitigate this.
• Trust Boundary – Third‑party orbs introduce external code. Always audit the orb source and verify signatures if available.
• Semantic Versioning Discipline – Teams must agree on version bumping rules to avoid accidental breaking changes.

Actionable Checklist for Your Team

1. Identify shared logic across pipelines (e.g., lint, test, deploy).
2. Define an orb with clear, documented jobs and commands.
3. Use circleci config validate and orb pack locally before publishing.
4. Publish the orb with a semantic version and review the changelog.
5. Consume the orb in projects with version pinning.
6. Implement automated tests that run the orb’s jobs in a sandbox CI job.
7. Maintain a version policy: patch for bug fixes, minor for new features, major for breaking changes.
8. Audit third‑party orbs and consider mirroring critical orbs in your own namespace.

By following these steps, your engineering teams can keep pipelines lean, enforce consistency, and reduce onboarding friction—all while staying aware of the overhead involved in maintaining custom orbs.