Creating and Verifying a Reproducible Spack Environment

Desired outcome

Create a named Spack environment that contains a specific set of packages, activate it in the current shell, and record the exact concretization in spack.lock so the same environment can be reproduced later.

Prerequisites

• Spack version 0.19 or newer (check with spack --version).
• A working compiler configuration (spack compilers shows at least one usable compiler).
• Write access to the Spack var directory ($SPACK_ROOT/var/spack/environments).
• Basic familiarity with Spack spec syntax (e.g., hdf5@1.12.1%gcc@11.2.0).

Procedure

1. Create the environment

   Choose a name (e.g., myenv) and optionally provide a spack.yaml file that lists the desired packages. If you do not have a yaml file, you can create the environment empty and add specs later.

   spack env create myenv

   This creates a directory $SPACK_ROOT/var/spack/environments/myenv containing an initial spack.yaml.

2. Activate the environment

   Activation modifies the shell’s SPACK_ENV variable so subsequent Spack commands operate inside the environment.

   spack env activate myenv

   After activation, the prompt may show the environment name depending on your shell configuration.

3. Add packages to the environment

   Specify the packages you need. For example, to add HDF5 and OpenMPI with specific versions and compilers:

   spack add hdf5@1.12.1%gcc@11.2.0
   spack add openmpi@4.1.5%gcc@11.2.0

   Each spack add updates the environment’s spack.yaml.

4. Concretize and install

   Run the installation. The --fail-fast flag stops at the first error, making it easier to detect problems.

   spack install --fail-fast

   Spack will resolve dependencies, write the concrete spec tree to spack.lock, and build/install the packages into the environment’s install path.

5. Generate module files

   Create a shell‑sourceable file that sets up the environment’s PATH, LD_LIBRARY_PATH, etc., for the installed packages.

   spack loads --to-file env_modules.sh

   This writes a script env_modules.sh in the current directory.

6. Load the module file

   Source the generated script to make the packages available in the shell.

   source env_modules.sh

Expected checks

• Verify loaded packages: spack find --loaded should list only the packages you added and their dependencies.
• Environment status: spack env status reports a clean environment with no mismatches between spack.yaml and spack.lock.
• Load a specific package: spack load hdf5 (or any added package) should succeed without error.
• Check the lock file: After installation, compare the current concretization with the lock using spack env status --diff. No output indicates a successful, reproducible environment.
• Validate a binary or library: Run a simple check, e.g., which h5cc or pkg-config --modversion openmpi, to confirm the expected version is present.

Recovery options

• Deactivate: Return to the default Spack context with spack env deactivate.
• Remove the environment: If the environment is no longer needed, delete it (including its spack.lock and install tree) with spack env rm -y myenv.
• Clean stage directories: Free space used during builds with spack clean -a (use cautiously; this removes all staged source tarballs and build directories).
• Revert to a previous lock: If you have a backup of spack.lock (e.g., spack.lock.backup), copy it back over the current lock and reinstall: cp spack.lock.backup spack.lock followed by spack install --fail-fast.

Limitations and practical verification

Environment isolation relies on activation; shells started outside the activation will not automatically see the environment’s packages unless you manually source the module file or use spack load. Changing the compiler or target architecture after environment creation may trigger reconcretization, potentially altering installed versions. To avoid surprises, keep the same spack.yaml (or regenerate the lock) when you change compilers or architectures.

Before relying on lock‑file reproducibility, confirm your Spack version supports it:

spack --version

Ensure the output is 0.19 or newer. Older versions lack full lockfile support, and reproducibility guarantees are weaker.