Warehouse: pypi.org shows latest version, pip see's previous version

[martinohanlon]

Describe the bug
I have recently uploaded a new version of the guizero package 0.5.1 (https://pypi.org/project/guizero/) which is being displayed as available on pypi.org.

However pip is not reporting the new version as being available, only the previous 0.5.0.

I have tried across multiple platorms (windows, mac, linux) all with the same result.

I have tried upgrading pip (already latest version), with no effect.

As far as I can tell everything has been packaged correctly and pypi.org shows no errors.

Expected behavior
I am expecting guizero 0.5.1 to be available to install via pip.

To Reproduce

Run:

pip3 install guizero

or

pip search guizero

v0.5.0 is reported as the latest version.

My Platform
I have tested on:
+Windows 10, pip version 10.0.1
+MacOS, pip version 10.0.1
+Raspbian, pip version 9.0.1

[martinohanlon]

Some additional information, the new vesion was uploaded using twine v1.11.0, setuptools 38.5.1.

[martinohanlon]

Ok, so this is getting weirder...

So now if I can install the guizero package which I couldnt do before.

pip3 install guizero --upgrade

... it installs the latest version

but if I run pip3 search guizero it still reports the latest version as being the previous version.

C:\Users\Martin O'Hanlon-LT>pip3 search guizero
guizero (0.5.0)  - Python module to allow kids to easily create GUIs
  INSTALLED: 0.5.1
  LATEST:    0.5.0

Do releases take time to propagate through warehouse / pypi?

[martinohanlon]

And now pip is reporting that version 0.5.1 is the latest...

C:\Users\Martin O'Hanlon-LT>pip3 search guizero
guizero (0.5.1)  - Python module to allow learners to easily create GUIs
  INSTALLED: 0.5.1 (latest)

Timing issues? Propagation of data? Something else?

[ewdurbin]

Hi @martinohanlon, thanks for the report.

Our search index is not updated incrementally yet, instead it is refreshed once every 3 hours. This would certainly lead to the behavior you're reported.

As such, I believe this is a duplicate of the feature request we are tracking in #701.

[martinohanlon]

@ewdurbin Its certainly seems to be a symptom of not having #701 .

I suppose if this is currently considered to be normal behaviour...

[ewdurbin]

We certainly want to implement incremental search index updates! Initial attempts in #3693 seemed to aggravate another issue, #3746

The pycon sprints are happening over the next four days and search issues are something we are going to try to prioritize.

[di]

To be clear, since pip uses the /simple API, which is instantaneously updated, you should be able to pip install your package as soon as you have uploaded it.

It's only pip search which depends on the search index, which is currently updated every 3 hours (as @ewdurbin said).

[martinohanlon]

I just did a couple of tests on this because what @di said didnt match with my experience when I created this issue.

This is what I did / found:

• I created a new packaged and uploaded it to pypi
• As soon as I uploaded it, as @di said, I was able to install it
• I then incremented the version number and uploaded it to pypi
• At this point I was unable to install the new version using pip install package --upgrade as it only found the previous version
• I uninstalled the packaged using pip uninstall package and tried to install it, but again it would only find the previous version
• I was only able to install the new package by disabling the cache using pip --no-cache-dir install package

In summary, providing you have never installed the packaged before, or you disable the cache, you can install a package as soon as its been uploaded, otherwise its dependent on the search index being updated.

[di]

I think what you're hitting here is that pip's local HTTP cache is not stale enough to go out and actually hit PyPI the second time. I don't think any point you're interacting with the search index.

This might be worth raising on the pip issue tracker, but at first glance I don't see any way to avoid this without using --no-cache-dir as it's just a side effect of caching.

[martinohanlon]

When I first raised this issue I saw this behaviour on multiple machines pip install package --upgrade would not identify the recently loaded version of a package. pip hadn't been used on these machines for several weeks, and as soon as the search index had been updated I was then able to run pip install package --upgrade successfully.

I'll raise an issue on the pip repo and link to this.

[rschwiebert]

@ewdurbin This is happening for me right now for mimeparse==0.1.4, which appears on pypi having been released Jan 2, 2013, and yet I'm having trouble with it in 2019. pip==19.1 reports 0.1.2 and 0.1.3 as being available, but it doesn't list or allow me to install 0.1.4.

Also, pip search does appear to be aware of 0.1.4.

It looks like the project just doesn't have any files for that version. Is that supposed to be possible? Are entries without installation files included intentionally, or is it likely an accident?

[jamadden]

mimeparse 0.1.4 has no files on PyPI (note the absence of the "Download files" link on the right of https://pypi.org/project/mimeparse/0.1.4/), so pip has no way to install it.

I can see how the search result would be a bit confusing.

[caramdache]

I have the issue with data-importer 3.1.1. It can be seen by pip3 search, but not by pip3 install which only sees data-importer 3.0.2.

I tried the -no-cache and -no-cache-dir and rm -rf ~/.cache/pip and upgraded pip, but still the issue is here.

I had to install from git:

pip3 install git+https://github.com/valdergallo/data-importer.git

[jamadden]

data-importer 3.1.1 only provides a Python 2 wheel. In contrast, 3.0.2 only provides a source distribution. Python 3 will not be able to install a Python 2 wheel, so 3.1.1 will not be used. 3.0.2's source distribution can be used by any version of Python so that's what Python 3 will be stuck with.

[caramdache]

Thanks @jamadden, I've reported this to the misconfigured package.

[valdergallo]

@caramdache I made the upload for p3k today. Thanks for helping me @jamadden and @caramdache

[amandal1810]

Python 3.8.3 (64-bit), Windows 10

pip cannot see latest version from pypi.org

screenshot from pypi.org

[di]

@amandal1810 This is likely due to https://github.com/protocolbuffers/protobuf/issues/7520