Python: pip install hangs on versions after and including 3.9.0

[jamesmealing]

When running pip to install packages on versions after and including 3.9.0, pip seems to hang and package installs take up to 20 minutes. This issue doesn't occur with version 3.8.6 or earlier.

[wglambert]

Can you give all the commands you ran and any relevant files or logs for reproducing the issue

I can't seem to reproduce

$ time docker build -t python:test - << EOF
FROM python:3.9.0
RUN pip install sh pyprind crypto
EOF

Sending build context to Docker daemon  2.048kB
Step 1/2 : FROM python:3.9.0
 ---> dfc47c6cee13
Step 2/2 : RUN pip install sh pyprind crypto
 ---> Running in 36c035e011c1
Collecting sh
  Downloading sh-1.14.0-py2.py3-none-any.whl (40 kB)
Collecting pyprind
  Downloading PyPrind-2.11.2-py3-none-any.whl (8.6 kB)
Collecting crypto
  Downloading crypto-1.4.1-py2.py3-none-any.whl (18 kB)
Collecting shellescape
  Downloading shellescape-3.8.1-py2.py3-none-any.whl (3.1 kB)
Collecting Naked
  Downloading Naked-0.1.31-py2.py3-none-any.whl (590 kB)
Collecting requests
  Downloading requests-2.24.0-py2.py3-none-any.whl (61 kB)
Collecting pyyaml
  Downloading PyYAML-5.3.1.tar.gz (269 kB)
Collecting chardet<4,>=3.0.2
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
  Downloading urllib3-1.25.10-py2.py3-none-any.whl (127 kB)
Collecting certifi>=2017.4.17
  Downloading certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
Collecting idna<3,>=2.5
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
Building wheels for collected packages: pyyaml
  Building wheel for pyyaml (setup.py): started
  Building wheel for pyyaml (setup.py): finished with status 'done'
  Created wheel for pyyaml: filename=PyYAML-5.3.1-cp39-cp39-linux_x86_64.whl size=542361 sha256=c68b26a54a6b39aa1983e96522b8d72ffb499087e0ee398ecea3142109296932
  Stored in directory: /root/.cache/pip/wheels/69/60/81/5cd74b8ee068fbe9e04ca0d53148f28f5c6e2c5b177d5dd622
Successfully built pyyaml
Installing collected packages: sh, pyprind, shellescape, chardet, urllib3, certifi, idna, requests, pyyaml, Naked, crypto
Successfully installed Naked-0.1.31 certifi-2020.6.20 chardet-3.0.4 crypto-1.4.1 idna-2.10 pyprind-2.11.2 pyyaml-5.3.1 requests-2.24.0 sh-1.14.0 shellescape-3.8.1 urllib3-1.25.10
Removing intermediate container 36c035e011c1
 ---> ca62cc29b5de
Successfully built ca62cc29b5de
Successfully tagged python:test

real    1m1.217s
user    0m0.080s
sys     0m0.048s

However when I tried installing pandas as an example I did encounter the repeated Installing build dependencies: still running... which had issues opened at https://github.com/pandas-dev/pandas/issues/32045 and https://github.com/pandas-dev/pandas/issues/36296

[sam-mathews]

It looks to me like this issue was present in the precommit check, which ran for 6 hours before being cancelled.

See: https://github.com/docker-library/python/runs/1217186972

[tianon]

Unfortunately, that was more likely to be https://www.githubstatus.com/incidents/t34640ccrmfs (it never even got around to running pip in that build -- we have several Write-Host instructions that would've printed output first if it had). :disappointed:

[sam-mathews]

I would be keen to know if @jamesmealing is encountering this issue locally or on Github Actions.

[NeuroFox]

Still happens to me. 3.6.9 is ok, 3.8 ok, at that very exact moment i switch to 3.9 it hangs during spacy installation.

You can check it more detailed here
https://github.com/explosion/spaCy/issues/6228

[jamesmealing]

Hi guys. Sorry for the slow reply.

I first noticed this issue running the image in a Concourse CI task:

Interestingly, the package I happened to use for testing was pandas so, I don't know if this could be related to the issue @wglambert referenced above... 🧐

This is the job I was running to recreate the issue:

platform: linux
image_resource:
  type: docker-image
  source: {
    repository: python,
    tag: 3.9.0 # or newer
    }
run:
  path: pip
  args: ["install", "pandas"]

By changing the 'tag' to '3.8.6' the issue went away.

Let me know if there is any further information or detail you want.

[ulgens]

I encountered the same issue today. Not sure how to debug that but that "hang" is around 3-4 minutes here and it's frustrating.

[wglambert]

Are you also using pandas or another extension that might not have everything compatible yet with 3.9? Can you give all the commands you ran and any relevant files or logs for reproducing the issue.

It seems there's still some updating to be done on a few extensions or for their dependencies, and I don't think there's anything actionable we as maintainers of the image could do to alleviate these issues

[jamesmealing]

Ah okay, so the issue is with certain third-party packages having not yet been updated to support the latest image version?

[wglambert]

Came across this support matrix for packages that support Python 3.9 https://pyreadiness.org/3.9/

Going to close since this isn't an issue with the image