Poetry Dependency Groups vs Extras: Choosing the Right Mechanism for Optional Dependencies

The Decision: Internal Tooling vs Public Features

Poetry 1.2 introduced tool.poetry.group as a native way to organize dependencies that should never reach PyPI. The older tool.poetry.extras mechanism remains the only path for optional features that downstream consumers select at install time via pip install pkg[feature]. The choice isn't mutually exclusive—most projects need both—but putting a dependency in the wrong category either bloats your published metadata or breaks consumer installs.

Quick Comparison

Aspect	Dependency Groups	Extras
Published to PyPI metadata	No	Yes (Provides-Extra)
Installable via pip install pkg[extra]	No	Yes
Poetry install command	poetry install --with group / --only group	poetry install --extras extra
Typical use cases	test, lint, docs, type-checking, dev tooling	redis, postgres, s3, cli, async
Visible in poetry export by default	No (requires --with)	Yes (with --extras)
Version introduced	Poetry 1.2.0 (2022)	Poetry 1.0

Trade-offs in Practice

Groups keep your wheel clean

When you run poetry build, only [tool.poetry.dependencies] and [tool.poetry.extras] appear in the generated METADATA / PKG-INFO. A [tool.poetry.group.test] containing pytest, hypothesis, and pytest-cov never reaches end users. This avoids the classic problem where pip install yourpkg pulls in test runners because they were listed in a dev extra.

Extras are the contract with consumers

If your library offers optional Redis support, declare it as an extra:

[tool.poetry.extras]
redis = ["redis-py"]

Consumers then run pip install yourpkg[redis] or poetry add yourpkg[redis] and get the right transitive dependencies. Groups cannot fulfill this role—pip doesn't understand them.

Mixing both is the normal pattern

A typical pyproject.toml ends up with three layers:

[tool.poetry.dependencies]
python = "^3.10"
requests = "^2.31"

[tool.poetry.group.test]
pytest = "^7.4"
pytest-cov = "^4.1"

[tool.poetry.group.lint]
ruff = "^0.1"
mypy = "^1.7"

[tool.poetry.extras]
redis = ["redis-py"]
async = ["aiohttp"]

Main dependencies are always installed. Groups are opt-in for contributors. Extras are opt-in for users.

Concrete Validation Steps

Create a throwaway project to verify the behavior before committing to a structure:

1. poetry new demo && cd demo
2. Edit pyproject.toml to add the group and extra sections shown above.
3. poetry lock — inspect poetry.lock; you'll see [metadata.groups] and [metadata.extras] sections recording each.
4. poetry install --only main then poetry show — confirm pytest, ruff, redis-py, and aiohttp are absent.
5. poetry install --with test,lint — verify dev tools appear.
6. poetry install --extras redis — verify redis-py appears.
7. poetry build && pip install dist/*.whl[redis] in a clean venv — confirm the extra works through pip.

Common Pitfalls

• Putting runtime deps in groups only. If redis-py lives only in [tool.poetry.group.redis], pip install yourpkg[redis] will fail because the extra is empty. Runtime-optional deps must be in an extra (or main deps).
• Exporting requirements.txt without groups. poetry export -f requirements.txt omits groups by default. CI jobs that consume the exported file need poetry export --with test,lint -f requirements.txt.
• Version conflicts across groups/extras. Poetry resolves a single version per package across all selected groups and extras. If test requires requests<3 and async extra requires requests>=3, resolution fails when both are selected.
• Older Poetry versions. Teams pinned to Poetry <1.2 will ignore [tool.poetry.group] sections entirely. Upgrade or keep dev deps in a dev extra as a fallback.

Migration Checklist

Moving from a single dev extra to groups:

1. Create [tool.poetry.group.test], [tool.poetry.group.lint], etc., and move dependencies out of extras.dev.
2. Delete or shrink extras.dev.
3. Update CI: replace poetry install --extras dev with poetry install --with test,lint,docs.
4. Update contributor docs: poetry install --with test,lint for full dev env, poetry install --only main for minimal.
5. Run poetry check and poetry lock --no-update to verify syntax and lock consistency.

Limitations to Accept

• Groups are Poetry-only. Any workflow that uses pip directly (including pip install -e . without Poetry) cannot install group dependencies.
• No per-group source selection. Private repository credentials in [tool.poetry.source] apply globally.
• poetry export cannot produce multiple requirements files for different group combinations in one pass; you must run it repeatedly with different --with flags.

Verification Checklist Before Merge

• poetry check passes (syntax valid).
• poetry lock --no-update succeeds (lock file consistent).
• poetry install --only main installs only runtime deps.
• poetry install --with test,lint installs dev tooling.
• poetry install --extras redis,async installs optional features.
• poetry build && pip install dist/*.whl[redis] works in a clean venv.
• Exported requirements.txt (with --with flags) matches CI expectations.

If all seven pass, your group/extra split is sound and won't surprise contributors or consumers.