Using Poetry Dependency Groups to Build Deterministic, Lean Production Images

Decision: Separate Dependencies into Groups or Keep Them All Together?

When you ship a Python application, the size of the runtime image and the consistency of the environment across developers, CI, and production are critical. Poetry offers dependency groups (previously called extras) that let you split packages into logical buckets – dev, test, docs, etc. The decision is whether to keep everything in the default group or to isolate non‑production packages into separate groups and install only the needed ones in the final image.

Constraints and Decision Criteria

• Deterministic Builds – All developers and CI pipelines must end up with the exact same set of packages.
• Image Size & Attack Surface – Production images should be as small as possible and contain only the runtime dependencies.
• Developer Experience – Developers should be able to install everything locally with a single command.
• CI Performance – Lockfile resolution should not become a bottleneck.

Supported Options

Option	Description	Pros	Cons
1️⃣ All in Default Group	All packages are listed under dependencies in pyproject.toml.	Simple configuration; one command to install everything.	Production image contains dev/test tools; larger size and higher attack surface.
2️⃣ Separate Groups (Recommended)	Use group sections: dev, test, docs etc. Production installs poetry install --no-dev.	Lean production images; deterministic lockfile per group; clear separation.	Requires extra commands for local dev; slight overhead in lockfile size.
3️⃣ Optional Extras	Define extras that can be enabled on demand.	Fine‑grained optional dependencies.	Can complicate lockfile resolution; less commonly used for dev/test separation.

Trade‑Off Analysis

• Determinism – All options generate a poetry.lock that pins transitive dependencies. Option 2 keeps the lockfile tidy by separating groups, making it easier to regenerate only the needed parts.
• Image Size – Option 1 includes pytest, black, etc. in the final image, inflating it by ~50 MB on average. Option 2 removes those, cutting size by 30‑70 MB depending on your stack.
• Security – Fewer packages mean fewer CVEs in the runtime environment. Grouping also limits the surface for supply‑chain attacks.
• CI Speed – Lockfile resolution is only done once per major change. Option 2 may take a few extra seconds to resolve the dev group, but the final poetry install --no-dev is usually faster because it pulls a smaller set of wheels.
• Developer Experience – Developers can run poetry install to get everything. Production pipelines can run poetry install --no-dev to stay lean.

Concrete Implementation

1. Define Groups in pyproject.toml

[tool.poetry]
name = "my‑app"
version = "0.1.0"
description = "Sample app"
authors = ["Jane Doe <[contact removed]>"]

[tool.poetry.dependencies]
python = "^3.11"
flask = "^2.3"

[tool.poetry.group.dev.dependencies]
black = "^23.3"
pre-commit = "^3.3"

[tool.poetry.group.test.dependencies]
pytest = "^7.4"
pytest‑cov = "^4.0"

[tool.poetry.group.docs.dependencies]
mkdocs = "^1.4"

Place the groups after [tool.poetry.dependencies]. Poetry 2.x supports the group syntax; Poetry 1.x uses extras but the same idea applies.

2. Install All Packages Locally

# Run as a regular user; no root required
poetry install

This pulls the default group plus dev, test, and docs into the virtual environment. Verify with:

poetry show --tree

You should see the full dependency tree, including all group packages.

3. Build a Production Image

# Create a minimal image
python -m venv .venv
source .venv/bin/activate
poetry install --no-dev

The --no-dev flag tells Poetry to skip the dev and test groups. The resulting environment contains only flask and its transitive dependencies.

4. Verify Determinism Across Machines

# On Machine A
poetry lock
poetry install --no-dev
poetry show --tree > a.txt

# On Machine B
poetry lock
poetry install --no-dev
poetry show --tree > b.txt

# Compare hashes
sha256sum a.txt b.txt

If the hashes match, the lockfile ensured identical environments. The poetry.lock file contains exact versions for all transitive dependencies, preventing drift.

Potential Risks and Mitigation

• Large Lockfile – With many groups, the lockfile can grow. Mitigate by keeping groups focused and removing unused packages.
• Conflict on Adding a Package – A new dependency may require a newer version of an existing transitive package, causing a resolver conflict. Resolve by running poetry lock and reviewing the conflict messages, then adjusting version constraints.
• Version Differences Between Poetry 1.x and 2.x – The group syntax is only available in Poetry 2.x. For older projects, use extras or upgrade to 2.x.

Practical Checklist

1. Run poetry lock after any change to pyproject.toml.
2. Use poetry install --no-dev for CI and Docker builds.
3. Periodically run poetry show --tree to audit the dependency graph.
4. Verify the lockfile hash across environments to catch drift.

Conclusion

Separating dependencies into logical groups and installing only the production group gives you deterministic builds, smaller images, and a cleaner security posture without sacrificing developer convenience. The trade‑offs are manageable, and Poetry’s lockfile mechanism guarantees that every environment matches exactly.