Pip: Python 3.8.2/pip3: Upgrading/Installing packages locally fails unable to find setuptools

[jkliegel]

Environment

• pip version:
• Python version:
• OS:

Description

Expected behavior

How to Reproduce

1. Get package from '...'
2. Then run '...'
3. An error occurs.

Output

Paste the output of the steps above, including the commands themselves and
pip's output/traceback etc.

[jkliegel]

Environment

pip version:
[jkliegel@ussjcs6208 el7]$ pip3 --version
pip 19.2.3 from /home/tools/VeriSilicon/Suites/VSTE/2019_Q2/el7/lib/python3.8/site-packages/pip (python 3.8)

Python version:
[jkliegel@ussjcs6208 el7]$ python3 --version
Python 3.8.2

OS:
[jkliegel@ussjcs6208 el7]$ uname -a
Linux ussjcs6208 3.10.0-862.el7.x86_64 #1 SMP Wed Mar 21 18:14:51 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux

Description

I am trying to install python + cocotb. I am behind a corporate firewall so have no direct internet access.

I downloaded python 3.8.3 source as python.org tarball distribution. Compiled, installed, everything seems to work.

Next step is to install python-dev tools (cocotb requirement). So trace dependencies, download source package (all from pypi.org) tarballs and start to install.

Original state:

[jkliegel@ussjcs6208 el7]$ pip3 list
Package    Version
---------- -------
pip        19.2.3 
setuptools 41.2.0 

Want to upgrade/install pip, setuptools, wheel:

[jkliegel@ussjcs6208 Repos]$ pip3 install --upgrade ./setuptools-46.1.3
Processing ./setuptools-46.1.3
Installing collected packages: setuptools
  Found existing installation: setuptools 41.2.0
    Uninstalling setuptools-41.2.0:
      Successfully uninstalled setuptools-41.2.0
  Running setup.py install for setuptools ... done
Successfully installed setuptools-46.1.3
[jkliegel@ussjcs6208 el7]$ pip3 list
Package    Version
---------- -------
pip        19.2.3 
setuptools 46.1.3 

So far so good:

[jkliegel@ussjcs6208 Repos]$ pip3 install --upgrade ./pip-20.0.2
Processing ./pip-20.0.2
  Installing build dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: /home/tools/VeriSilicon/Suites/VSTE/2019_Q2/el7/bin/python3.8 /home/tools/VeriSilicon/Suites/VSTE/2019_Q2/el7/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-9jng962l/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel
       cwd: None
  Complete output (8 lines):
  Collecting setuptools
    WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3aaeb58580>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
    WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3aaeb582e0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
    WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3aaeb58340>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
    WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3aaeb660a0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
    WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3aaeb66220>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
    ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
  ERROR: No matching distribution found for setuptools
  ----------------------------------------
ERROR: Command errored out with exit status 1: /home/tools/VeriSilicon/Suites/VSTE/2019_Q2/el7/bin/python3.8 /home/tools/VeriSilicon/Suites/VSTE/2019_Q2/el7/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-9jng962l/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel Check the logs for full command output.

Expected behavior

Expected pip to be able to see installed setuptools. Obviously I do not understand something. Pointers to documentation (RTFM) or good Python Systems Administration books would be welcome.

How to Reproduce

See above.

Thank you for your time and effort on my behalf

Sincerely,

James

[gutsytechster]

Can you reformat the traceback by enclosing it within three backticks?

Apart from it, on having a look at the traceback, you seem to have some network connection problem. I am not sure if it would resolve the issue, but can you try re-installing the packages after toggling your internet connection(or maybe with some other connection)

[jkliegel]

Here is reformatted output:

[jkliegel@ussjcs6208 Repos]$ pip3 install --upgrade ./pip-20.0.2
Processing ./pip-20.0.2
Installing build dependencies ... error
ERROR: Command errored out with exit status 1:
command: /home/tools/VeriSilicon/Suites/VSTE/2019_Q2/el7/bin/python3.8 /home/tools/VeriSilicon/Suites/VSTE/2019_Q2/el7/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-9jng962l/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel
cwd: None
Complete output (8 lines):
Collecting setuptools
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3aaeb58580>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3aaeb582e0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3aaeb58340>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3aaeb660a0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3aaeb66220>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
ERROR: Could not find a version that satisfies the requirement setuptools (from versions: none)
ERROR: No matching distribution found for setuptools

ERROR: Command errored out with exit status 1: /home/tools/VeriSilicon/Suites/VSTE/2019_Q2/el7/bin/python3.8 /home/tools/VeriSilicon/Suites/VSTE/2019_Q2/el7/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-9jng962l/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel Check the logs for full command output.

[jkliegel]

Actually not having the network connection is correct. I am in a secure environment with no network connection. Therefore I am bringing distributions (tarballs obtained from python.org and pypi.org) across onto secure machines and installing there.

This works fine for the python v3.8.2 distribution: compiles and installs correctly, runs tests, works as expected.

The problem starts when I try to use pip3 to install or upgrade packages from local distributions.

I first try to install/upgrade setuptools, pip, and wheel.

pip3 install --upgrade ./setuptools-46.1.3

seems to work as expected. Note the "./" to cause pip3 to install from a local distribution.

However when I next try to upgrade pip with

pip3 install --upgrade ./pip-20.0.2

(Note the "./" denoting local distribution)

pip3 can not find the upgraded/installed setuptools. It then tries to go find the setuptools and can not find a network connection (no surprise) and dies.

But the root problem is that it can not find setuptools after the first upgrade.

Thanks

James

[McSinyx]

Is it a PEP 518 thing, i.e. it tries to install build dependencies in a virtual environment?

[xavfernandez]

@jkliegel by default with a project containing a pyproject.toml pip will try to build the package "in isolation" (and therefore needs to install setuptools in this temporary isolated environment.

Since you are offline, you have 2 options:

• use --no-build-isolation to disable the isolation mechanism
• provide --find-links=directory_containing_setuptools_wheel so that pip can still install setuptools in the temporary isolated environment.

[jkliegel]

Thank you for you help. The solution that worked for me was

pip3 install --upgrade --no-build-isolation ./setuptools-46.1.3
pip3 install --upgrade --no-build-isolation ./wheel-0.34.2
pip3 install --upgrade --no-build-isolation ./pip-20.0.2
pip3 list
Package Version

---

pip 20.0.2
setuptools 46.1.3
wheel 0.34.2