Pip: Wheels not installing properly

[brl0]

Environment

• pip version: 19.2.3, 19.3.1
• Python version: 3.5-3.8
• OS: Windows, Linux, MacOSX

Description

When installing a wheel from PyPI (and TestPyPI), there seems to be inconsistent behavior as to when it installs *.so and *.pyd files. I can consistently recreate the issue in Azure pipelines, but not on my machine.

Expected behavior

It should install all appropriate files, at least those that match extensions that result from this command:
python -c "import importlib.machinery; print(importlib.machinery.all_suffixes())"

How to Reproduce

1. Get package from '...'
   python -m pip install -vvv --no-cache-dir --ignore-requires-python --force-reinstall -I --prefer-binary --no-clean --disable-pip-version-check --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple primesieve

2. Then run '...'
   python -c "from primesieve import primes;print(primes(10))"

3. An error occurs.

Output

ModuleNotFoundError: No module named 'primesieve._primesieve'

To help troubleshoot, I put in a try/except to print the file names in the module directory, which shows that some of the expected files are missing.

[chrahunt]

Can you please provide a link to the Azure pipelines logs?

[brl0]

I hope these work for you.

Here is a link to the logs on one of the jobs:
https://dev.azure.com/brl0/7771c36c-ed65-405d-b649-617a018b68f5/_apis/build/builds/106/logs/125

Here is a link to try to download all the logs for that run:
https://dev.azure.com/brl0/7771c36c-ed65-405d-b649-617a018b68f5/_apis/build/builds/106/logs?$format=zip

[chrahunt]

I think the issue is caused by the current working directory being automatically added to sys.path ahead of the site packages directory, which is a normal Python behavior. Since it contains the sources (e.g. primesieve/__init__.py), this gets resolved as the one and only package directory for primesieve instead of the proper one in site-packages.

Notice that in the output of pip show, the installed location is the site-package directory:

2019-12-22T23:13:45.3451312Z Name: primesieve
2019-12-22T23:13:45.3451712Z Version: 2.1.3
...
2019-12-22T23:13:45.3468516Z Location: /opt/hostedtoolcache/Python/3.5.7/x64/lib/python3.5/site-packages

but in the traceback at the end, it is the work folder:

2019-12-22T23:13:45.5694955Z Traceback (most recent call last):
2019-12-22T23:13:45.5695421Z   File "<string>", line 1, in <module>
2019-12-22T23:13:45.5695720Z   File "/home/vsts/work/1/s/primesieve/__init__.py", line 14, in <module>
2019-12-22T23:13:45.5696064Z     raise error
2019-12-22T23:13:45.5696328Z   File "/home/vsts/work/1/s/primesieve/__init__.py", line 12, in <module>
2019-12-22T23:13:45.5696652Z     from ._primesieve import *
2019-12-22T23:13:45.5697305Z ImportError: No module named 'primesieve._primesieve'

This is not an uncommon problem, and one of the justifications for using a "src" layout like that shown in pypa/sampleproject (or as used in pip).

Can you try changing directory to be somewhere that doesn't contain the sources before executing the test command?

[brl0]

You, sir, are a genius! That is a great catch. Thanks!