Spack Modulefiles vs. a Hand-Maintained Module Tree: Choosing an Exposure Strategy

The decision, stated plainly

Spack installs each package into a prefix whose directory name encodes a hash of the concretized spec. That is good for reproducibility and bad for humans: nobody wants to type a 32-character hash into PATH. So the real question is not whether to expose Spack installs, but which layer owns the environment.

There are three supported answers: let Spack generate modulefiles and let the site module loader read them, hand-write modulefiles that point at Spack prefixes, or skip modules entirely and export variables by hand. This guide compares them against the constraints that usually decide the outcome, then walks through a validation loop you can run in a scratch prefix before touching production.

Three constraints that actually decide it

• Which loader the site already runs. Lmod reads Lua modulefiles; classic Environment Modules reads Tcl. Spack can emit either, plus Python, but a loader silently ignores a format it does not parse. This constraint is not negotiable — it eliminates options before any other consideration.
• Whether modulefiles must be identical across nodes. If login nodes and compute nodes mount the same shared prefix, one generated tree serves everyone. If they do not, you need a synchronization story, and generated files are easier to regenerate than hand-written ones are to re-audit.
• How much manual curation the team can sustain. Hand-written modulefiles are pleasant until the first Spack upgrade changes a dependency's install path. Then every file referencing that path is stale.

Comparing the options

Approach	What it produces	Strength	Cost or limit
Spack-generated Tcl modulefiles	Files consumed by classic Environment Modules	Mirrors the concretized dependency graph; regenerable	No hierarchical modulepaths; naming conventions are yours to define
Spack-generated Lua modulefiles	Files consumed by Lmod	Works with Lmod's hierarchy and module spider discovery	Hierarchy configuration is site-specific and easy to get wrong
Spack-generated Python modulefiles	Files consumed by loaders that parse Python	Useful where the site standardizes on a Python-based loader	Least common target; verify your loader before committing
Hand-written modulefiles over Spack installs	A curated tree you own	Full control over names, defaults, and user-facing text	Drifts from the install tree; every Spack change is a manual edit
Manual PATH/LD_LIBRARY_PATH exports	Shell snippets or dotfiles	Zero infrastructure; fine for a single user debugging one build	Does not scale, and silently diverges from the spec over time

The middle ground — Spack generates, the site loader consumes — is usually the right default because it keeps one source of truth. Hand-written files are justified mainly when you need user-facing behavior Spack's templates do not express, and even then the file should reference the Spack prefix rather than duplicate its contents.

Trade-offs worth naming out loud

Format coupling. Choosing Lua commits you to Lmod; choosing Tcl commits you to Environment Modules. Migrating loaders later means regenerating everything, which is cheap — but only if you did not hand-edit the generated files.

Hierarchy. Lmod's hierarchical modulepaths (compiler, then MPI, then application) give users a much smaller module avail listing. Spack has configuration for hierarchical Lmod generation, but the naming scheme is a site convention, and a scheme that reads well on one cluster can collide on another where a package name is already taken.

Shadowing. Two Spack instances, or two installs of the same package, can emit modulefiles with the same name onto the same modulepath. Whichever the loader finds first wins, and the loser becomes invisible. This is the most common silent failure in mixed deployments.

A concrete path: generate into a scratch prefix, then validate

Run these on the host that owns the Spack instance, as the account that owns the install tree. You need write access to the module prefix you configure — not to the Spack install prefix, which should already be read-only in production.

Spack's module subcommands and configuration keys have changed across releases. Confirm the syntax for your installed version before scripting anything:

spack --version
spack module --help
spack config get modules   # shows the keys your release actually honors

Then point generation at a scratch directory and enable one format. The key names below are version-dependent — treat them as a starting point and confirm against spack config get modules:

spack config add 'modules:default:enable:[tcl]'
spack config add 'modules:default:prefix:/scratch/$USER/modtest'
spack module tcl refresh --help   # check the non-interactive flag for your release
spack module tcl refresh -y zlib  # restrict to one spec while testing

Expected check: files appear under /scratch/$USER/modtest with the extension your loader parses (.tcl for Environment Modules, .lua for Lmod). If the directory is empty, the format and the loader disagree, or the prefix key was not honored.

Now validate in an interactive shell. module is a shell function, so it must be evaluated in the shell you are testing — not inside a script that never sourced the loader's initialization:

module use /scratch/$USER/modtest
module avail zlib
module load zlib
module show zlib          # inspect path-prepend and environment-set entries
which zlib-flate          # substitute a real executable from the package
echo "$LD_LIBRARY_PATH"

Read the module show output against the spec's dependencies: every prepend-path should point into a Spack install prefix, not into a system directory. Then run a small program linked against the loaded package on a target compute node, because a successful login-node load does not prove the shared prefix is mounted where the job actually runs.

Verification checklist

1. Generated files exist at the configured modulepath with the expected extension.
2. module avail lists the package exactly once — a duplicate means shadowing.
3. module show paths resolve to Spack prefixes.
4. The package's executable resolves via which after loading.
5. A linked program runs on a compute node, not just the login node.

Limitations and rollback

Generation writes to a shared prefix, so it changes state. Do not point it at the production modulepath until the scratch validation passes. If a regeneration goes wrong, the recovery is to restore the previous module tree — keep a copy, or generate into a versioned directory and switch the modulepath entry that points at it. After regenerating in place, users may need to reload the module system because some shells and schedulers cache the module index.

Finally, confirm write permissions and ownership on the module prefix for the account running generation, and re-check the exact subcommand names against your Spack release before wiring this into automation. The commands above are illustrative of the workflow, not verified output from a specific version.