Managing Development and Test Dependencies with Poetry Groups

Problem: Accidentally shipping dev tools to production

When a Python project lists all dependencies under [tool.poetry.dependencies], running poetry install on a server pulls in linters, test frameworks, and other development‑only packages. This bloats the image, slows startup, and can introduce version conflicts.

Thesis: Poetry dependency groups let you declare separate sets of packages and install only the ones you need for a given environment.

Defining groups in pyproject.toml

Poetry 1.0+ treats dependency groups as first‑class citizens. Add a block like:

[tool.poetry.dependencies]
python = "^3.9"
requests = "^2.31"

[tool.poetry.group.dev.dependencies]
black = "^24.0"
isort = "^5.0"

[tool.poetry.group.test.dependencies]
pytest = "^8.0"
pytest-cov = "^5.0"

The [tool.poetry.dependencies] section is the "main" group that is always installed. Each additional group lives under tool.poetry.group..dependencies. Markers (# comments) are optional; you can also use environment markers directly on a line if you need conditional inclusion.

Installing with groups

From the project root, run:

• poetry install – installs only the main group (production).
• poetry install --with dev – adds the dev group.
• poetry install --with dev,test – adds both dev and test groups.
• poetry install --only main – equivalent to the plain install, useful in CI to be explicit.

These commands require read access to the pyproject.toml and poetry.lock files and the ability to create a virtual environment in ~/.cache/pypoetry/virtualenvs (or a location set by POETRY_VIRTUALENVS_PATH). No elevated privileges are needed unless you install system‑wide with poetry install --system, which we advise against for isolation.

After installation, you can verify what was pulled in:

poetry show -G dev

This lists the packages and versions that belong to the dev group. Compare the output with the versions declared under [tool.poetry.group.dev.dependencies] to confirm correctness.

Keeping the lock file in sync

Whenever you add, remove, or change a version in any group block, run:

poetry lock

This updates poetry.lock with the exact resolved versions for all groups. If you only want to ensure the lock file reflects the current pyproject.toml without attempting to fetch newer versions, use:

poetry lock --no-update

Running poetry lock is safe; it only writes to the lock file and does not modify installed environments.

To check that your lock file is consistent, execute:

poetry show --tree

Look for any dev‑only packages appearing under the main tree when you have run poetry install (without --with). If you see them, the lock file is stale and you need to re‑run poetry lock.

Worked example

Suppose you start with a minimal library:

[tool.poetry.dependencies]
python = "^3.9"
requests = "^2.31"

[tool.poetry.group.dev.dependencies]
black = "^24.0"
isort = "^5.0"

[tool.poetry.group.test.dependencies]
pytest = "^8.0"
pytest-cov = "^5.0"

1. Run poetry lock to create the initial lock file. 2. In a CI job that builds wheels, execute poetry install --only main. Verify with poetry show -G dev that no output appears (or that the command returns an empty list). 3. In a local development shell, run poetry install --with dev,test. Then poetry show -G dev should list black and isort, and poetry show -G test should list pytest and pytest-cov. 4. If you later add flake8 = "^7.0" to the dev group, run poetry lock again before committing.

Trade‑off and limitation

The main benefit is clear separation of concerns, but the mechanism relies on Poetry being the installer. If your CI pipeline uses pip install . or a custom script that calls pip install -r requirements.txt, Poetry groups are ignored and all declared dependencies end up installed. To avoid this, either keep the Poetry‑only workflow or generate a requirements file from the lock with poetry export -f requirements.txt --without-hashes and ensure the export excludes groups you don’t want (e.g., --without dev --without test).

Another practical limitation is that the lock file can become out of sync if you forget to re‑run poetry lock after editing groups. The verification step (poetry show -G <group>) helps detect drift, but it adds a small manual step to your workflow.

Actionable closing

To start using Poetry groups today:

1. Add [tool.poetry.group..dependencies] blocks for each concern (dev, test, docs, etc.).
2. Run poetry lock to record the resolved versions.
3. In production or CI, install with poetry install --only main (or plain poetry install).
4. In development, install the groups you need with poetry install --with dev,test.
5. Verify with poetry show -G <group> and re‑run poetry lock whenever you edit the groups.

Following these steps keeps your production images lean, your lock file reliable, and your dependency intentions explicit.