Question
Poetry dependency resolver backtracking during lock file generation
Zo Indigo
0 reputation · 03 Nov 2023, 06:16 UTC
85.1K views1
Loading…
0 reputation · 03 Nov 2023, 06:16 UTC
Poetry uses a deterministic resolver and a backtracking algorithm to generate the poetry.lock file based on the constraints defined in pyproject.toml. While this ensures environment consistency, the resolution process can encounter performance bottlenecks when managing deeply nested dependency trees or restrictive version constraints.
There is uncertainty regarding the specific trigger points that cause the resolver to enter extensive backtracking cycles, particularly when stable versions are available but conflict with secondary dependencies.
31775 reputation · 03 Nov 2023, 07:35 UTC
The most significant backtracking delays are usually caused by tight, intersecting version ranges that leave few or no viable versions for a package, especially when combined with extras or platform markers that create many conditional dependency sets. The resolver will then try many combinations before concluding a conflict or finding a solution.
When pre‑releases exist but are not explicitly requested, Poetry’s resolver prefers stable versions** and only considers a pre‑release if **no stable version satisfies all constraints**. Pre‑releases are effectively filtered out unless the constraint syntax itself allows them (e.g., a caret or explicit range that includes a prerelease identifier).
poetry==1.2.3) combined with a dependent that requires a different exact pin.^1.0, ~1.0) that are further restricted by another dependency’s !=* exclusion or a narrow range.; sys_platform == 'win32') that create separate dependency trees; if each tree has conflicting constraints, the resolver must explore many marker combinations.poetry lock -vvvpoetry show --tree to view its dependents and the constraints they impose.==1.2.3 to ^1.2.3 or >=1.2.0,<2.0.0).pyproject.toml, run poetry lock again and confirm the verbosity output shows far fewer attempts and a swift completion.poetry.lock files to ensure only the intended version changes occurred.If the verbosity log does not clearly point to a single package, please share the excerpt of the poetry lock -vvv output showing the repeated attempts; that will let us pinpoint the exact constraint pattern causing the backtracking.
Use comments to ask for clarification. Post a solution as an answer.
4,210 reputation · 03 Nov 2023, 09:14 UTC
Poetry’s resolver switched to the PubGrub algorithm in 1.2, which is depth‑first but still explores alternatives when a dependency’s version set becomes empty. The most common triggers are:
<=X.Y.Z than the root, forcing earlier packages to downgrade.>=1.0,<2.0 intersected with >=1.5,<1.8 makes the resolver iterate through all compatible releases before finding a fit.By default Poetry filters out any pre‑release candidates unless the root constraint explicitly allows them (e.g., ^1.2.0-rc1) or the user runs poetry install --pre. This means pre‑releases are only considered after stable releases fail to satisfy all constraints.
Run poetry lock --verbose on a deliberately conflicted project to watch the backtracking log. The output shows each package version tried and why a branch was abandoned. Also, setting POETRY_EXCLUDE_OPTIONAL=1 will skip optional dependencies entirely, potentially reducing resolution time.