Poetry: Poetry doesn't handle transitive dependencies correctly if they are Python version dependant

Created on 11 May 2020  路  4Comments  路  Source: python-poetry/poetry

  • [x] I am on the latest Poetry version.
  • [x] I have searched the issues of this repo and believe that this is not a duplicate.
  • [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
  • OS version and name: macOS 10.15.4
  • Poetry version: 1.0.5

Issue


Let's say you have package_A that depends on package_B, which depends on different version's of package_C depending on Python's version, like so:

package_A's pyproject.toml:

[tool.poetry.dependencies]
python = "^2.7 || ^3.5"
package_B = ["*"]

package_B's setup.py (I've come across this problem where the dependency uses setup.py, but there shouldn't be a difference if the dependency uses Poetry):

setup(name='package_B',
      install_requires=[
          "package_C>=0.0.50, <2.0; python_version<'3.0'",
          "package_C>1.0.0; python_version>'3.0'"
      ]
)

Now, if you try to poetry install package_A, package_C will not get installed along with the other dependencies, but the installation still might be successful, because when Poetry installs the root package after the dependencies, it issues pip install -e, which will install package_C then. However, if package_C is in a private package index, the installation will fail as the --extra-index-url is not used to install the root package, and pip will not find the package on PyPi.

What happens under the hood is, that Poetry will try to solve the dependency graph first for Python 2 and in order to save resources it will cache packages in memory. This package object stores its requirements in a list (.requires). package_C will be a duplicate in this list for package_B. When it solves the graph the incompatible one will get removed, and the new, cleaned dependency list will be put back to the package's .requires field without cloning the original package, therefore the cached package's requirements will be overwritten as well.

image

image

By the next iteration when Poetry tries to solve for Python 3, it will find an incomplete requirement list for package_B, and package_C will not get installed.

Solution

Cloning the package object around here should solve the problem.

Bug Triage

Most helpful comment

I think I have a similar issue. Let me know if I should file it separately or not.
The package django-heartbeat defines a python2 dependecy that is being included in my poetry.lock file even though I am locked to python 3.6.10
https://github.com/pbs/django-heartbeat/blob/6639b78fafcdd7ead50ad5cbebefc626e6c2e556/setup.py#L16

[[package]]
category = "main"
description = "Your project's heartbeat/healthcheck and dependency status"
name = "django-heartbeat"
optional = false
python-versions = "*"
version = "2.0.2"

[package.dependencies]
Django = ">=1.6"
psutil = "4.0.0"
py2-ipaddress = ">=3.4.0"

All 4 comments

I think I have a similar issue. Let me know if I should file it separately or not.
The package django-heartbeat defines a python2 dependecy that is being included in my poetry.lock file even though I am locked to python 3.6.10
https://github.com/pbs/django-heartbeat/blob/6639b78fafcdd7ead50ad5cbebefc626e6c2e556/setup.py#L16

[[package]]
category = "main"
description = "Your project's heartbeat/healthcheck and dependency status"
name = "django-heartbeat"
optional = false
python-versions = "*"
version = "2.0.2"

[package.dependencies]
Django = ">=1.6"
psutil = "4.0.0"
py2-ipaddress = ">=3.4.0"

Same here. FYI @stephsamson @sdispater

and here

@peterdeme @psaghelyi Could you try with the latest prerelease of poetry 1.1.0a3?

Was this page helpful?
0 / 5 - 0 ratings