Poetry Dependency Groups & Lock File: Reproducible Python

Learn how Poetry’s dependency groups and lock file mechanism give you isolated, reproducible Python environments. A step‑by‑step example shows adding groups, locking, and exporting to requirements.txt.

Problem

When a Python project grows, keeping the runtime exactly the same on every machine becomes a nightmare. Traditional requirements.txt files let you pin versions, but they’re easy to forget, and the install process can still pull in unexpected updates if the lock file is out‑of‑sync. Even worse, developers often mix production and development dependencies in a single list, making it hard to isolate the core library from test tools or optional extras.

Thesis

Poetry’s dependency groups and poetry.lock file together form a robust, deterministic workflow. By scoping dependencies into groups and letting Poetry resolve and pin every package in the lock file, you get isolated, reproducible environments without the maintenance overhead of manually syncing requirements.txt files.

Understanding Dependency Groups

Poetry supports four built‑in groups: main, dev, test, and optional. You can also create custom groups when you need a finer granularity.

main – Packages required for the application to run in production.
dev – Tools for development (e.g., linters, formatters).
test – Test frameworks and helpers.
optional – Extras that consumers may enable via pip install mypkg[extra].

Defining groups keeps the environment clean: running poetry install --no-dev installs only the main group, while poetry install --with dev,test pulls everything you need for local development.

The Poetry.lock Mechanism

After you declare dependencies, Poetry resolves them by contacting the configured package index (default PyPI). The resolution algorithm is deterministic: it starts with the highest‑level constraints (e.g., main) and works its way down, respecting group boundaries and extras. The result is written to poetry.lock.

Key features of the lock file:

**Exact version pins** – every entry contains the full name, version, and a SHA‑256 hash of the wheel or source archive.
**Source isolation** – Poetry never falls back to other indices unless explicitly configured, preventing accidental upgrade from a different source.
**Deterministic builds** – cloning a repository and running poetry install will always pull the same package files.

Inspecting the Lock File

Open poetry.lock and you’ll see sections like:

[[package]]
name = "requests"
version = "2.32.0"
hash = "sha256:..."

Each package also lists its own dependencies, creating a complete dependency graph.

Worked Example

Below is a minimal project that demonstrates groups and lock file usage. Assume you’re on a fresh clone of the repo.

**Create a new project**
```
poetry new myapp
cd myapp
```
**Add a production dependency**
```
poetry add flask
```
**Add a development dependency**
```
poetry add --group dev black
```
**Add a test dependency**
```
poetry add --group test pytest
```
**Add an optional extra**
```
poetry add --group optional redis
```
**Generate the lock file**
```
poetry lock
```

**Export to a legacy requirements file (optional)**

poetry export -f requirements.txt --without-hashes -o requirements.txt

After the poetry lock step, poetry.lock contains entries for flask, black, pytest, and redis, each with a hash. Running poetry install --no-dev will install only flask and its dependencies; running poetry install --with dev,test pulls the rest.

Trade‑offs & Limitations

Lock File Drift: If you add or update a dependency but forget to run poetry lock, the lock file stays stale. The next poetry install will still use the old version, breaking reproducibility. A common mitigation is to add poetry lock --no-update to CI pipelines after every poetry add.
Migration from requirements.txt: Extras and group constraints don’t map 1:1. Converting a flat requirements.txt to Poetry often requires manual inspection to decide which packages belong in dev or test, and whether they should be optional.
Performance on Large Projects: The resolution step can be slow for very large dependency graphs, though Poetry’s algorithm is optimized for speed.

Take‑away Actions

**Always run poetry lock after adding or updating a dependency** to keep the lock file current.
**Use group flags** (--group dev, --group test) to keep the runtime environment lean.
**Export for legacy tools**: poetry export -f requirements.txt --without-hashes gives you a quick snapshot for CI tools that still consume requirements.txt.
**Verify reproducibility**: On a clean machine, run poetry install and compare the installed package list with the one in poetry.lock. A quick sanity check is poetry show --tree.

By embracing Poetry’s groups and lock file, you gain deterministic builds, cleaner environments, and a clear separation between what your application needs to run and what developers need to work on it.

Mastering Poetry: Dependency Groups & Lock Files for Reproducible Python Environments