Failed to concretize spec: unsatisfiable constraints – concretizer nondeterminism in Spack

Question

Goal: obtain a Spack environment that concretizes to the same concrete spec every time, regardless of when it is built or which Spack version is used, so that builds are repeatable across teams and CI pipelines.

Concern: the default concretizer does not guarantee deterministic output; the resulting concrete spec can vary with the ordering of packages in spack.yaml or with internal solver heuristics, and the solver backend changed between Spack 0.19.x and 0.20.x, leaving determinism as an optional feature.

Questions: What configuration yields deterministic concretization without losing solver completeness? How does package ordering in spack.yaml influence the concretizer’s decision process? Is the clingo‑based concretizer mature enough for production‑grade reproducibility?

Zoha · Answer

Answer

To obtain a Spack environment that concretizes to the same concrete spec every time, use the clingo concretizer and ensure that any ambiguous versions or variants are explicitly pinned. This configuration preserves solver completeness while eliminating the nondeterminism that causes the “unsatisfiable constraints” error.

How to configure deterministic concretization

Set the concretizer to clingo in spack.yaml (or ~/.spack/config.yaml):
```
concretizer:
  clingo: {}
```
Disable parallel solving to avoid any hidden nondeterminism from thread scheduling:
```
concretizer:
  clingo:
    threads: 1
```
Pin any packages that the concretizer reports as ambiguous (see step 2 below) by adding a concrete version or variant to the spec, e.g.
```
spec:
  - myapp@=1.2.3 ^dependency@=4.5.6
```

Why package ordering matters with the classic concretizer

The classic concretizer processes packages in the order they appear in spack.yaml. As it assigns versions and variants, early choices propagate constraints to later packages. If a later package cannot satisfy those propagated constraints, the solver backtracks and may explore a different branch. Because the search order depends on the input ordering, the final concrete spec can vary between runs or across Spack versions, leading to nondeterministic outcomes.

The clingo concretizer encodes the entire problem as a logic program and lets the solver find a stable model independent of input order, making the result deterministic.

Maturity of the clingo‑based concretizer

Since its introduction in Spack 0.19, the clingo concretizer has been used in production CI pipelines and is considered stable. It provides complete search (it will report unsatisfiability only when no solution exists) and deterministic output, satisfying the reproducibility requirements for most teams.

Steps to diagnose the current failure

Enable concretizer debugging to see the conflict graph:
```
spack concretize --debug <spec>
```
Locate the line that reports a conflict or unsatisfiable constraint (look for “conflict”, “unsatisfiable”, or “no viable version”). Note the exact package name and the version/variant that is causing the trouble.
Either relax that constraint (e.g., allow a broader version range) or add a concrete pin for the conflicting package/variant.
Re‑run spack concretize --debug <spec>; the message should disappear and a concrete spec will be produced.

Verify the spec installs correctly:

spack install <spec>
spack find --loaded

Missing diagnostic detail

To give a precise pinning recommendation, I need the exact package name and version/variant that the debug log identifies as conflicting. Please provide that information so I can suggest the minimal change needed to resolve the unsatisfiability.