pip list --outdated misses old version of tf-nightly

[bersbersbers]

Environment

• pip version: 20.0.2
• Python version: 3.8.2
• OS: Linux

Description
pip list --outdated misses old version of tf-nightly. I have tf-nightly==2.2.0.dev20200227 installed, and a newer version is available and can be installed.

Expected behavior
pip list --outdated should show tf-nightly.

Output

user@host:~> pip list | grep tf-nightly
tf-nightly                     2.2.0.dev20200227
user@host:~> pip list -u | grep tf-nightly
user@host:~> pip list -o | grep tf-nightly
user@host:~> pip --version
pip 20.0.2 from /home/user/.pyenv/versions/3.8.2/lib/python3.8/site-packages/pip (python 3.8)
user@host:~> pip list -o -v | grep tf-nightly | grep -v Skip
1 location(s) to search for versions of tf-nightly:
* https://pypi.org/simple/tf-nightly/
Fetching project page and analyzing links: https://pypi.org/simple/tf-nightly/
Getting page https://pypi.org/simple/tf-nightly/
Looking up "https://pypi.org/simple/tf-nightly/" in the cache
https://pypi.org:443 "GET /simple/tf-nightly/ HTTP/1.1" 304 0
  Found link https://files.pythonhosted.org/packages/db/80/3d2a67fbe6818b140f8c53a5a4e9ed16683fa77fcebb453e006fb75ecd50/tf_nightly-2.2.0.dev20200227-cp38-cp38-manylinux2010_x86_64.whl#sha256=aaf5ce7f469754681354480f710d2a89004ee624fd7186f7e6f21b4a2a26c3bc (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200227
  Found link https://files.pythonhosted.org/packages/59/9c/972de8fb6246be6557a16565c4cc1977ea9e275540a77ec4a2e0057dc593/tf_nightly-2.2.0.dev20200228-cp38-cp38-manylinux2010_x86_64.whl#sha256=6273178b4fe579a5e5a22edd412b114b47bd2e5b90e37814335995602a9733e6 (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200228
  Found link https://files.pythonhosted.org/packages/8e/1f/b7c76f80e3ecd403c7b4e3124be455e9b01c013a6555c6dc215c53130d1f/tf_nightly-2.2.0.dev20200229-cp38-cp38-manylinux2010_x86_64.whl#sha256=3bf6be5dd2c72743869fbf03ddc11f86155df0ccc74f0666c978ebc2becd530c (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200229
  Found link https://files.pythonhosted.org/packages/45/c5/80ce0b8519be89cfaf8c7be41dd7ed8f19e89082417f474eb212ad4ecb9f/tf_nightly-2.2.0.dev20200301-cp38-cp38-manylinux2010_x86_64.whl#sha256=0b20b993c239f6eb4b7e44f68677c2923b889af0f62fbecb0ae0af3c9865ec90 (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200301
  Found link https://files.pythonhosted.org/packages/f8/a5/7815c0f20f1535a0731f427de14cc7f60f881ee7db4f81c5f067da613abc/tf_nightly-2.2.0.dev20200302-cp38-cp38-manylinux2010_x86_64.whl#sha256=a5c13dcc3e1955e1eb76aaf442a4844293b1e80a8640cb47f01b1ae957dabfea (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200302
  Found link https://files.pythonhosted.org/packages/39/52/2747c5049c03b9a51e258c718a6d53721b48669ea1ec495643362559f8ae/tf_nightly-2.2.0.dev20200303-cp38-cp38-manylinux2010_x86_64.whl#sha256=5fd99bd84925a48d432cf6ea351bb689809ca2f13b68330020f5088b4b141164 (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200303
  Found link https://files.pythonhosted.org/packages/f4/70/94cdec35f3de1013e1d12f03dc8dc21624c6b6276444ba62972831785511/tf_nightly-2.2.0.dev20200304-cp38-cp38-manylinux2010_x86_64.whl#sha256=783fde7be577e6affdfbce4802ef0803b52336623e5ee4284c97c85bc7789c34 (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200304
  Found link https://files.pythonhosted.org/packages/45/fa/eb30478fde21cb94cc0e5ce84c57100fb53eace2b8da66a84cfacd67acd9/tf_nightly-2.2.0.dev20200305-cp38-cp38-manylinux2010_x86_64.whl#sha256=7a9203fa1f1e48c16b3d54a6d4d93657505fc7c8139a7fd35c6b38c68eefd8c0 (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200305
  Found link https://files.pythonhosted.org/packages/be/a7/c5f14bddbcc29ad99b69756ab21f3ddf5e64633a54b5b4929c51337d1fab/tf_nightly-2.2.0.dev20200306-cp38-cp38-manylinux2010_x86_64.whl#sha256=bb3b16ee01cf59f9e40a077f46c6f9398cd329fb6173cd7f521670f3fb70d583 (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200306
  Found link https://files.pythonhosted.org/packages/30/8e/650b282491e126a39e2b256f1ad790d566c53465dfd801a7703e421431e3/tf_nightly-2.2.0.dev20200307-cp38-cp38-manylinux2010_x86_64.whl#sha256=b9d0c045d29438a089ba0f2a40aef43c1c6435387d66ee80835f84bf26ed1557 (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200307
  Found link https://files.pythonhosted.org/packages/34/72/1f66dfcadc68762d5310288795269a55c2730f1d1067d86cff489548ae50/tf_nightly-2.2.0.dev20200308-cp38-cp38-manylinux2010_x86_64.whl#sha256=1f12208da4de572f7387d2eaf7c93c6fff3edda62359e70e6c013d500ec50251 (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200308
  Found link https://files.pythonhosted.org/packages/9c/f5/a7af0a53e764fc2702b13cba704b3ec8f8017416ea572a567f8c956bf3e9/tf_nightly-2.2.0.dev20200309-cp38-cp38-manylinux2010_x86_64.whl#sha256=26912aa7ca3984381deec965d4029f614dcc1ad80880e40d1aed8ecd4e4027f5 (from https://pypi.org/simple/tf-nightly/), version: 2.2.0.dev20200309
user@host:~> pip uninstall tf-nightly
Found existing installation: tf-nightly 2.2.0.dev20200227
Uninstalling tf-nightly-2.2.0.dev20200227:
  Would remove:
    /home/user/.pyenv/versions/3.8.2/bin/estimator_ckpt_converter
    /home/user/.pyenv/versions/3.8.2/bin/saved_model_cli
    /home/user/.pyenv/versions/3.8.2/bin/tf_upgrade_v2
    /home/user/.pyenv/versions/3.8.2/bin/tflite_convert
    /home/user/.pyenv/versions/3.8.2/bin/toco
    /home/user/.pyenv/versions/3.8.2/bin/toco_from_protos
    /home/user/.pyenv/versions/3.8.2/lib/python3.8/site-packages/tensorflow/*
    /home/user/.pyenv/versions/3.8.2/lib/python3.8/site-packages/tf_nightly-2.2.0.dev20200227.dist-info/*
Proceed (y/n)? y
  Successfully uninstalled tf-nightly-2.2.0.dev20200227
user@host:~> pip install tf-nightly
Collecting tf-nightly
  Downloading tf_nightly-2.2.0.dev20200309-cp38-cp38-manylinux2010_x86_64.whl (517.0 MB)
     |████████████████████████████████| 517.0 MB 25 kB/s
(Requirement already satisfied: ...)
Installing collected packages: tf-nightly
Successfully installed tf-nightly-2.2.0.dev20200309

[pfmoore]

You probably need --pre to tell pip list --outdated to consider pre-release versions.

[uranusjr]

I wonder if pip should use the same behaviour as pip install tf-nightly>=2.2.0.dev20200227 here. I’d assume the user wants to consider prereleases if the installed version is a prereleases. But as discussed in #7579, it’s difficult to tell definitely what is the most expected behaviour.

[bersbersbers]

@pfmoore yes, thank you!
@uranusjr I would definitely like that. Especially for a package like tf-nightly which has only prerelease version available, and nothing else. Maybe this could be a good starting point for an implementation.

[pfmoore]

pip install will pick up pre-releases if there are no full releases available (that behaviour is essentially described in PEP 440). That's why you didn't need to specify --pre when installing. Agreed it might be nice for pip list --outdated to somehow take a similar approach, but as @uranusjr noted, it's not at all obvious what the right behaviour should be in all the corner cases (it's usually obvious what makes sense in any particular case, but working out a general rule that pip can apply is far from easy - that's typical of any case of a "do what I mean" style of behaviour).

I suggest that working out precise behaviour should probably wait for someone to create a PR, as it's much easier to debate design choices with working code to ground the discussion.

[bersbersbers]

I suggest that working out precise behaviour should probably wait for someone to create a PR, as it's much easier to debate design choices with working code to ground the discussion.

@pfmoore, I find this to be a pretty unexpected (to say the least) proposal. I cannot think of any context where "do some coding first, then let's think if what the code is doing is actually what we want" is still a reasonable approach. Fortunately, I believe that under this premise, this issue will probably not attract enough attenuation for someone to potentially waste their time on this; unfortunately, this issue would remain unresolved as a consequence. Also unfortunately, though, I am probably not knowledgeable enough regarding pip to give much more meaningful input other than my expected behavior in the original bug description, but I'll try nonetheless.

I must admit I am not even seeing a problem with an approach such as

pip list --outdated should pick up pre-releases of package_a if a pre-release version of package_a is already installed.

(This formulation includes the case I mentioned, where there are only pre-release available, as pip --list only looks at installed packages - and if one is installed, it must be pre-release.)

Another formulation of a potential solution would be this:

Given package_a==v installed, would pip install package_a>v result in a newer package being installed? If so, call it outdated, otherwise, uptodate.

I believe this is the assumption I made when issuing pip --list outdated for the first time, also assuming that having tf-nightly==2.2.0.dev20200309 installed, pip install tf-nightly>2.2.0.dev20200309 would install tf-nightly==2.2.0.dev2020031, which it should according to PEP 440, but does not in line with #7579. (At least that's consistent, but I feel that's more an accident than based on my formulation above.)

Finally, another assumption I am holding dear is that a package is either uptodate or outdated. In my example above, tf-nightly appears neither in the pip list -u nor in the pip list -o, and regardless of which one it should be exactly, it needs to be (exactly) one of them in my opinion.

[pfmoore]

I find this to be a pretty unexpected (to say the least) proposal. I cannot think of any context where "do some coding first, then let's think if what the code is doing is actually what we want" is still a reasonable approach.

Fair point. Let me put it another way. I personally don't have any intuition on what would be a reasonable thing to do here. So I can't really comment on what would be a good design. I suspect that many other people would be in the same situation. So, if someone (you, for example) wants to see this feature in pip, they should probably just write a PR and make what seem like reasonable choices based on their usage.

Once we have a PR, my experience is that people who previously had no real opinion, find that reviewing the PR makes them come up with questions like "hey, what about XXX?" (actually, writing the PR also tends to do that!) and that will generate additional feedback, which may result in design changes.

I hope that explains better.

I must admit I am not even seeing a problem with an approach such as

pip list --outdated should pick up pre-releases of package_a if a pre-release version of package_a is already installed.

I can't immediately see an issue with this.

Another formulation of a potential solution would be this:

Given package_a==v installed, would pip install package_a>v result in a newer package being installed? If so, call it outdated, otherwise, uptodate.

Does that give the same resuluts (in all cases) as the previous suggestion? I can't immediately convince myself that it will. If it doesn't, which one would you propose?

Finally, another assumption I am holding dear is that a package is either uptodate or outdated

That makes sense as an invariant. But it doesn't explain why tf-nightly doesn't satisfy it. So I'm not sure how it helps the discussion here, until you determine the reason for that.

As you can see, I have questions, but no real opinions here 🙂

[bersbersbers]

I hope that explains better.

Yes, it does - at least as I understand it, the assumption is that a PR will probably be accepted unless someone comes up with reasonable doubts that it's a good solution. Makes sense to me.

Does that give the same results (in all cases) as the previous suggestion? I can't immediately convince myself that it will. If it doesn't, which one would you propose?

I don't know, as I know too little about the behavior of pip install or pip list -o. In particular, with pip list -o, it is not even defined in the documentation what outdated means. Just imagine that someone understands outdated as "more than a year old" - little in the documentation would convince that person that this is not the definition used by pip list.

Finally, another assumption I am holding dear is that a package is either uptodate or outdated

That makes sense as an invariant. But it doesn't explain why tf-nightly doesn't satisfy it. So I'm not sure how it helps the discussion here, until you determine the reason for that.

I agree, it's a separate issue, and it's one I cannot explain convincingly because again, pip --list does not say what uptodate means. My guesswork is this:

tf-nightly does not have any stable release. Assuming that pip --list ignores pre-release versions, it basically ignores everything pypi has to offer, and from that it concludes that
1) tf-nightly is not outdated (because the latest [release] package is not newer [it does not exist])
2) tf-nightly is not uptodate (because the latest [release] package is not the installed version [it does not exist])

https://github.com/pypa/pip/blob/0f095af4ba7ac1af2c27024fa539af4c3a45d002/src/pip/_internal/commands/list.py#L157-L160

https://github.com/pypa/pip/blob/0f095af4ba7ac1af2c27024fa539af4c3a45d002/src/pip/_internal/commands/list.py#L164-L174

https://github.com/pypa/pip/blob/0f095af4ba7ac1af2c27024fa539af4c3a45d002/src/pip/_internal/commands/list.py#L182-L209

[bersbersbers]

I think, intuitively (not having tested it yet), I would replace

https://github.com/pypa/pip/blob/0f095af4ba7ac1af2c27024fa539af4c3a45d002/src/pip/_internal/commands/list.py#L189

by

if not (options.pre or dist.parsed_version.is_prerelease):

I know that's not technically a PR, but is that good enough for people to start discussing it?