CircleCI cache vs workspace: Node.js CI persistence

Problem: every job reinstalls node_modules

Your Node.js pipeline runs lint, build, and test as separate CircleCI jobs. Each job starts from a fresh container, so each one runs npm install or npm ci from scratch. As the dependency tree grows, those installs dominate pipeline time. The build also produces a dist/ folder that the test job needs, but the test job has no way to see it unless something carries it forward.

CircleCI offers two persistence tools that look similar but solve different problems: caches and workspaces. Picking the wrong one means either stale dependencies or missing build artifacts.

Thesis: cache dependencies, hand off artifacts with workspaces

A cache is a stored copy of files that CircleCI can restore in a later job, even across different workflows. Caches are best for immutable dependencies such as node_modules or the npm cache, keyed to a lockfile so they refresh when dependencies change.

A workspace is a temporary filesystem snapshot passed between jobs inside the same workflow. Use it for mutable build outputs, such as compiled dist/ directories, test reports, or generated configuration, that a downstream job must consume.

The lockfile (usually package-lock.json) records exact dependency versions. An executor is the environment where a job runs, such as a Docker container.

Worked example: build and test jobs

Add this to .circleci/config.yml in your repository. Replace cimg/node:20 with your standard Node executor image. The config runs on the CircleCI executor, not on your local machine, and requires checkout permission so jobs can read the repository.

version: 2.1
jobs:
  build:
    docker:
      - image: cimg/node:20
    steps:
      - checkout
      - restore_cache:
          key: deps-v1-{{ checksum "package-lock.json" }}
      - run:
          name: Install dependencies
          command: npm ci
      - save_cache:
          key: deps-v1-{{ checksum "package-lock.json" }}
          paths:
            - ~/.npm
            - node_modules
      - persist_to_workspace:
          root: .
          paths:
            - dist
  test:
    docker:
      - image: cimg/node:20
    steps:
      - checkout
      - attach_workspace:
          at: .
      - run:
          name: Verify workspace contents
          command: |
            ls -R dist
            du -sh dist
      - run:
          name: Run integration tests
          command: npm run test:integration
workflows:
  node-ci:
    jobs:
      - build
      - test:
          requires:
            - build

In the build job, restore_cache tries to reuse dependencies from a previous run. If the key matches, CircleCI skips most of npm ci. After installation, save_cache stores the result under the same lockfile-derived key. Then persist_to_workspace sends dist/ to the test job.

The test job uses attach_workspace so it can run integration tests against the build output without reinstalling dependencies. The requires: [build] rule ensures test only starts after build completes in the same workflow.

Trade-offs and limits

• Caches are best-effort. CircleCI can evict them, and a missing or poorly designed key causes a cache miss. Never omit the lockfile checksum from the key, or a stale cache may be restored after dependency changes.
• Cache keys use prefix matching. A fallback key such as deps-v1- can restore an older cache if the exact key misses, which is useful for speed but risks stale dependencies if the lockfile changed.
• Workspaces exist only within a single workflow run. A job in a later pipeline or a different workflow branch cannot attach to them.
• Workspaces count against plan-specific storage quotas, and transferring large directories between jobs can lengthen total pipeline time.
• Saving caches on every job consumes minutes and credits. Gate saves behind lockfile changes or explicit cache version bumps.

Verify before you rely on it

1. Open the build job in the CircleCI app and search the log for Restoring cache. Confirm whether the primary key hit or the job fell back to a cache miss.
2. Keep the Verify workspace contents step shown above. It prints dist contents and size, so you can confirm artifacts arrived intact before tests execute.
3. Modify package-lock.json on a feature branch and confirm the cache key changes. This forces a fresh save rather than a stale prefix hit.
4. Review your CircleCI plan documentation for workspace storage quotas and cache retention policies before depending on large artifacts.

Closing: classify, then measure

Split files into two categories: immutable dependencies that can be safely discarded and rebuilt, and mutable outputs that must flow to the next job. Cache the first group, workspace the second. Then watch real pipeline logs for cache hit rates and workspace transfer size before tuning keys or splitting jobs further.

If the pipeline breaks after merging this config, revert the commit that changed .circleci/config.yml. Caches and workspaces regenerate on the next run, so there is no separate state to roll back.