Poetry: Version matching does not conform to PEP440 when specifier sets contain pre-release versions

[charmasaur]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name:
• Poetry version: macOS Catalina 10.15.2 (19C57)
• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/charmasaur/6188c4c872b47da599e945691570ff31

Issue

Poetry version matching doesn't seem to conform to PEP440, specifically for specifier sets containing pre-release versions and inequality constraints.

According to https://www.python.org/dev/peps/pep-0440/#exclusive-ordered-comparison: "The exclusive ordered comparison

From the lock file generated from the pyproject.toml file in the gist above, we have:

<snip>
tensorflow-estimator = ">=2.1.0rc0,<2.2.0"
<snip>
[[package]]
category = "main"
description = "TensorFlow Estimator."
name = "tensorflow-estimator"
optional = false
python-versions = "*"
version = "2.2.0rc0"

That is, despite the constraint "<2.2.0" we're getting the version "2.2.0rc0".

The issue seems to be https://github.com/python-poetry/poetry/blob/master/poetry/semver/version_range.py#L63, which doesn't account for pre-release versions.

Incidentally, when trying to hunt down the cause of this issue I also came across https://github.com/python-poetry/poetry/blob/master/poetry/version/specifiers.py#L715, which doesn't seem to be used but does look to have a similar issue: if any of the specifiers in the set allow prereleases, the entire set is considered to allow prereleases, which will mean that a constraint like ">=2.1.0rc0,<2.2.0" is considered to allow prereleases and will thus erroneously allow "2.2.0rc0".

[az0uz]

I think the version constrain solver is at fault here:

from poetry.semver import parse_constraint, Version
parse_constraint("~=2.1.0").allows(Version.parse("2.2.0rc0"))

returns True
It seems poetry converts parse_constraint("~=2.1.0") to >=2.1.0,<2.2.0, which I guess is wrong because then we have 2.2.0rc0 < 2.2.0 (which is true)
parse_constraint("~=2.1.0") should be >=2.1.0, <2.2.0a0

[tomzx]

After some investigation, it appears the cause is in poetry-core dependency.py.

https://github.com/python-poetry/poetry-core/blame/master/poetry/core/packages/dependency.py#L46-L49

Since the minimum version is a prerelease (in your case 2.2.0rc0), then it converts the dependency into a "allow-prereleases" dependency which will let it install any prerelease version. In your case, the interesting thing is that any prerelease of 2.2.0 are also considered as valid as @az0uz explained.

Here it seems that prerelease should convert the constraint into one that is similar to what @az0uz suggested since what the "prerelease" property seems to want to let you do is install prereleases that would match the same version as the minimum constraint but not the maximum version specified.

In the original example, you either have to remove the rc0 bit (>=2,1,0,<2.2.0) so it will not consider any prereleases of both 2.1.0 and 2.2.0 or replace the constraint with one mentioned by @az0uz (>=2.1.0rc0,<2.2.0a0).

[vbersier]

Can reproduce with package numba.

numba (0.50.1) depends on llvmlite (>=0.33.0.dev0,<0.34) but then 0.34.0rc1 gets installed.

test code

from poetry.core.semver import parse_constraint, Version
range = parse_constraint('>=0.33.0.dev0,<0.34')
rcver = Version.parse('0.34.0rc1')
range.allows(rcver)  # returns True

The problem seems to be that https://github.com/python-poetry/poetry-core/blob/28848f06e3201d511815f8b00e25a77b2ca82d37/poetry/core/semver/version_range.py#L75 doesn't account for the version being a pre-release.