Reproducible Python Environments with conda: environment.yml, Exports, and Recovery

The problem: "works on my machine" environments

A conda environment built by running conda install commands over several weeks is nearly impossible to recreate by hand. The fix is to treat the environment as a declared artifact: a single environment.yml file that any teammate (or CI job) can rebuild with one command. This guide covers writing that file, choosing an export strategy, verifying the result, and recovering when an environment breaks.

Assumptions: conda 23.x or newer (Miniconda, Anaconda Distribution, or Miniforge), on Linux, macOS, or Windows. Commands run in a terminal (Anaconda Prompt on Windows) and need no elevated privileges.

Prerequisites

• conda installed and on your PATH. Check with conda --version.
• A decision on which channel your project standardizes on (covered below — this matters more than most people expect).
• Write access to your project directory; the yml file should live in version control next to your code.

Step 1: Write environment.yml by hand, not by export

The most reproducible starting point is a small, hand-written file listing only what your code directly imports:

name: data-pipeline
channels:
  - conda-forge
dependencies:
  - python=3.11
  - numpy=1.26
  - pandas>=2.1
  - pyarrow=14
  - pip
  - pip:
      - some-package-not-on-conda

Key decisions embedded here:

• One channel. Mixing defaults and conda-forge is a common source of solver conflicts andABI mismatches, because the same package can be built differently in each. Pick one (conda-forge is the common choice for open-source stacks) and stick to it. Reinforce it with conda config --set channel_priority strict, which tells the solver to prefer packages from higher-priority channels rather than picking newer versions from any channel.
• Pin majors, consider leaving minors flexible. numpy=1.26 pins the minor version; pandas>=2.1 leaves room. Hard-pinning everything maximizes reproducibility but can make the environment unsatisfiable months later as builds are retired. For a library under active development, flexible minor versions plus a lock file (below) is a pragmatic trade-off.
• pip section caution. Packages under pip: are installed after conda solves its own set, and pip does not solve against conda packages. Conflicts may only surface at runtime as import errors. Keep the pip section minimal.

Step 2: Create and verify the environment

From the directory containing the file:

conda env create -f environment.yml

Then run three checks:

1. conda env list — the new environment name should appear.
2. conda list -n data-pipeline — confirm installed versions match your pins.
3. Activate and import the critical packages:

   conda activate data-pipeline
   python -c "import numpy, pandas, pyarrow; print(numpy.__version__, pandas.__version__)"

   This catches missing shared libraries and ABI conflicts that a successful solve does not guarantee against.

If the solve is painfully slow, check which solver is active with conda config --show solver. Recent conda releases default to the much faster libmamba solver; on older installs you may need to opt in. Exact defaults are version-sensitive, so verify on your installation.

Step 3: Choose an export strategy for stronger reproducibility

A hand-written yml re-solves every time, so two builds a month apart can get different dependency versions. You have three options, in increasing strength:

A common team workflow: commit the hand-written environment.yml as the source of truth, plus a per-platform explicit spec for CI so builds resolve identical packages instead of re-solving.

Recovery when an environment breaks

These operations change state, so pick the least destructive first:

1. Roll back a bad install. conda records a revision history per environment. Run conda list --revisions -n data-pipeline to see them, then conda install -n data-pipeline --revision 3 to return to revision 3. This is the fastest fix for "it worked before my last install."
2. Clone a known-good environment. If a colleague's env works: conda create -n data-pipeline-fix --clone good-env.
3. Nuke and rebuild. conda env remove -n data-pipeline, then recreate from the yml. This is exactly why the yml file matters — deletion becomes a safe operation.

Limitations and how to validate the whole setup

• environment.yml does not capture system-level libraries (CUDA drivers, system compilers). Document those separately.
• Explicit spec files are platform-locked; maintain one per OS you support.

The definitive validation: build from your yml on a clean machine or fresh container (docker run --rm -v $PWD:/work -w /work condaforge/miniforge3 conda env create -f environment.yml), then run your test suite inside it. If that passes, your environment is genuinely reproducible rather than accidentally working.