Commit the Lock File: Pinning an HPC Stack with Spack Environments

You send a colleague the environment file for your cluster build — petsc %gcc@11.2.0 ^openmpi@4.1.5 — and a day later their simulation links against a different hypre than yours. Nothing in the file was wrong. A Spack environment file describes intent, not a complete build plan, and every fresh concretization is free to make its own choices for the dependencies you never named. The fix is procedural, not technical: run the concretizer once, commit the spack.lock it produces, and treat that lock file — not the YAML — as the artifact you share.

spack.yaml is intent; spack.lock is the plan

A Spack environment is a directory containing spack.yaml, a list of abstract specs: package names plus the constraints you care about. It says nothing about the many transitive dependencies PETSc pulls in — hypre, BLAS/LAPACK implementations, compression and math libraries, Python bindings.

Concretization fills that gap. Spack's solver (answer-set-programming based since the 0.16 release) assigns an exact version, compiler, variant set, and dependency edges to every package in the graph, honoring your constraints plus preferences from packages.yaml. The solved graph is written to spack.lock beside the YAML. The two files have different audiences: humans review the YAML, machines consume the lock.

One setting deserves a deliberate decision: concretizer:unify, which controls whether the environment is solved as one shared graph or as independent specs.

Setting	Behavior	Cost
unify: true	One shared solution; every spec uses the same dependency versions	Smallest, most shareable stack; fails if specs genuinely need conflicting versions
unify: when_possible (default)	Shares dependencies where compatible, splits where needed	Usually concretizes, but can duplicate heavy dependencies
unify: false	Each spec solved independently	Always finds a solution, but may build two MPIs or two BLAS libraries

For a team building one application stack, unify: true is usually what you want: a single openmpi and a single BLAS across everything, at the cost of a harder solve.

A worked example: PETSc with a pinned compiler and MPI

Run this on the machine where builds happen — typically a cluster login or build node, under your own account. Spack needs no root access; it installs into its own prefix. The example assumes Spack 0.21 or newer; flags and the lock-file schema vary between releases, so check spack --version and each command's --help for your checkout.

First confirm the compiler you are pinning is actually registered in Spack's configuration, because concretization fails if gcc@11.2.0 is unknown:

spack compiler list

Then create and populate the environment:

spack env create myenv
spack env activate -p myenv
spack add petsc %gcc@11.2.0 ^openmpi@4.1.5
spack concretize

spack add writes the spec into spack.yaml; the unify and view settings you add by hand, then re-run spack concretize:

spack:
  specs:
    - petsc %gcc@11.2.0 ^openmpi@4.1.5
  view: true
  concretizer:
    unify: true

Before building anything, inspect the solved graph: spack find -dl lists every concretized package with its hash, and spack find -d petsc prints dependency paths — confirm that openmpi@4.1.5 and %gcc@11.2.0 appear where you expect. (spack solve <spec> previews a solution without touching the environment, which is handy for experiments.) The view: true line additionally assembles a merged prefix of the installs that you can put on PATH.

Then build. This compiles the full DAG from source unless your site provides binaries via a mirror or buildcache, so budget real time:

spack install --fail-fast

Now the step that makes the exercise pay off — commit both files. That requires the environment to live in a repository directory, so create it with a path (spack env create myenv /path/to/repo); named environments live under Spack's own var/spack/environments prefix instead:

git add spack.yaml spack.lock
git commit -m "Pin PETSc stack: gcc 11.2.0, OpenMPI 4.1.5"

On another machine with the same Spack release, clone the repository, run spack env activate -p ./myenv, then spack install. Spack reads the lock and builds exactly the recorded graph — no re-solve, no drift. If you need to undo local state, spack env remove myenv deletes the environment, and because the lock is version-controlled, an unwanted re-concretization is a git checkout -- spack.lock away.

What the lock file does not buy you

• System-level software. spack.lock pins Spack packages, not your OS: glibc, kernel modules, drivers, and system CUDA runtimes sit outside the graph. The same lock on different distributions can still produce different binaries.
• Spack itself. The lock schema and solver behavior change between releases, so pin the Spack checkout (a release tag) alongside your environments, or a colleague's newer Spack may rewrite or reject the lock.
• Bit-for-bit builds. Identical plans plus identical compiler binaries yield identical installs; a differently built gcc of the same version can still change results.

And the cost side: solve time grows quickly with stack size, and large environments can take minutes to concretize. spack concretize -f re-solves from scratch and can churn the lock even when nothing meaningful changed — harmless if the lock is reviewed like any other build artifact, a silent hazard if it is not.

Make it a habit, not a rescue

On your next stack, set unify deliberately instead of living with the default, commit spack.yaml and spack.lock together, and record the Spack release that produced the lock. To verify the result end to end, run spack install --fail-fast from the committed lock in CI and diff spack find -dl output between machines — identical hash lists mean everyone is building the same stack, which was the point.