Which method is preferred for managing compiler overrides in shared HPC environments?

Question

Spack utilizes compiler.json files to store paths, flags, and versioning information for detected compilers. While spack compiler find automates the creation of these entries, HPC environments often feature compilers installed in non-standard locations or specific module-loaded paths that auto-detection may overlook.

In shared environments where multiple users require different compiler variants or specific flag overrides, there is uncertainty regarding the optimal balance between global configuration and user-specific overrides. Relying solely on spack compiler add can lead to fragmented configurations across different user home directories.

Does Spack support a hierarchical inheritance model for compiler.json to allow site-wide defaults with user-level overrides?
  What is the recommended approach for ensuring consistency in compiler specifications across a shared cluster without requiring every user to manually run detection?

Tasadduq Burney · Answer

Answer to the Question
The most robust approach in a shared HPC environment is to use Spack’s hierarchical configuration system: a site‑wide compilers.yaml that defines the baseline compiler set, followed by optional user‑level overrides and, for per‑project needs, environment‑level specifications.
Why This Works

Single source of truth for the cluster: administrators add compilers once with --scope=site or by editing /etc/spack/compilers.yaml.
Granular control for users: each user can add or tweak entries in ~/.spack/linux/compilers.yaml without affecting others.
Project isolation when needed: a spack.yaml in an environment can declare a compiler that shadows both site and user definitions.

Practical Steps

Admin adds site compilers:
spack compiler add --scope=site gcc@9.3.0
spack compiler add --scope=site intel@2023.1

Verify site visibility:
spack config show compilers
# Look for "scope: site" entries

Optional user overrides (per user, runs as that UID):
spack compiler add --scope=user gcc@10.2.0

Check user‑only visibility:
spack config show compilers
# Entries marked "scope: user" should appear only for that user

Project‑specific override (environment):
Create an spack.yaml in the project directory:
packages:
  all:
    compiler: [gcc@10.2.0]

Run spack spec -i mypkg to confirm the overridden compiler.

Things to Verify

All users can see the site compilers: spack config --show-scopes compilers should list site under scopes.
User overrides do not leak: run the same spack config show compilers as a different UID and confirm the user entry is absent.
Environment overrides take precedence: after adding a compiler in spack.yaml, spack spec should display that compiler in the spec output.

Missing Diagnostic Detail
Do all users load the same Spack installation (e.g., via a shared module or container)? If not, user‑level overrides may be ignored when a different Spack instance is used.

Which method is preferred for managing compiler overrides in shared HPC environments?

1 answer

Answer to the Question

Why This Works

Practical Steps

Things to Verify

Missing Diagnostic Detail

Your perspective belongs here.

0 question comments