Black: Black install via pip fails on embedded Python on Windows using latest pip version (20.3)
Describe the bug
A new version of pip has recently been released, which is a lot stricter in many ways. When installing the latest version of Black on a Windows machine using the embedded Python distribution, I get an error:
ERROR: Requested black from https://files.pythonhosted.org/packages/dc/7b/5a6bbe89de849f28d7c109f5ea87b65afa5124ad615f3419e71beb29dc96/black-20.8b1.tar.gz#sha256=1c02557aa099101b9d21496f8a914e9ed2222ef70336404eeeac8edba836fbea has different version in metadata: '0.0.0'
The package it is trying to download is the latest source release (20.8b1), and from what I can gather, pip finds that the version in the filename (20.8b1) conflicts with the version in the metadata inside the package - which it says is 0.0.0.
To Reproduce
- Install the embedded Python interpreter on Windows (https://docs.python.org/3/using/windows.html#the-embeddable-package)
- Install pip in the embedded Python interpreter (download https://bootstrap.pypa.io/get-pip.py and run it using the embedded Python interpreter's python.exe)
- Run
python.exe -m pip install black --no-cache-dir - You will see the following output:
c:\pepys-deploy\pepys-import>python -m pip install black --no-cache-dir
Collecting black
Downloading black-20.8b1.tar.gz (1.1 MB)
|鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅| 1.1 MB 3.3 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
ERROR: Requested black from https://files.pythonhosted.org/packages/dc/7b/5a6bbe89de849f28d7c109f5ea87b65afa5124ad615f3419e71beb29dc96/black-20.8b1.tar.gz#sha256=1c02557aa099101b9d21496f8a914e9ed2222ef70336404eeeac8edba836fbea has different version in metadata: '0.0.0'
If the latest version which has a wheel release published to PyPI is specified instead, then it works fine:
c:\pepys-deploy\pepys-import>python -m pip install black==19.10b0 --no-cache-dir
Collecting black==19.10b0
Downloading black-19.10b0-py36-none-any.whl (97 kB)
|鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅| 97 kB 1.1 MB/s
Requirement already satisfied: typed-ast>=1.4.0 in .\python\lib\site-packages (from black==19.10b0) (1.4.1)
Requirement already satisfied: toml>=0.9.4 in .\python\lib\site-packages (from black==19.10b0) (0.10.2)
Requirement already satisfied: regex in .\python\lib\site-packages (from black==19.10b0) (2020.11.13)
Requirement already satisfied: attrs>=18.1.0 in .\python\lib\site-packages (from black==19.10b0) (20.3.0)
Requirement already satisfied: appdirs in .\python\lib\site-packages (from black==19.10b0) (1.4.4)
Requirement already satisfied: pathspec<1,>=0.6 in .\python\lib\site-packages (from black==19.10b0) (0.8.1)
Requirement already satisfied: click>=6.5 in .\python\lib\site-packages (from black==19.10b0) (7.1.2)
Installing collected packages: black
WARNING: The scripts black.exe and blackd.exe are installed in 'c:\pepys-deploy\pepys-import\python\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed black-19.10b0
So, it seems to be something to do with the metadata checking before building the wheel which is going wrong.
The latest version of Black worked fine with the previous version of pip which didn't use the new strict metadata checker.
I can't seem to reproduce this in non-embedded Python, but it may be an issue there too. I'm aware that pip isn't officially supported for embedded Python on Windows, but it is widely used and all other packages seem to install fine:
Expected behavior
Expected to install successfully.
Environment (please complete the following information):
- Version: 20.8b1
- OS and Python version: Windows embedded Python 3.7
Does this bug also happen on master?
N/A as related to PyPI releases
robintw
All 8 comments
/me boots Windows VM to debug tonight or some time soon hopefully
cooperlees
on 30 Nov 2020
Just as an additional piece of information: the install works when when you tell pip to use it's legacy resolver which isn't as strict about metadata (see below). However, you can see that it is definitely confused about versions - it states Successfully installed black-0.0.0 at the end.
c:\pepys-deploy\pepys-import>python -m pip install black --no-cache-dir --use-deprecated=legacy-resolver
Collecting black
Downloading black-20.8b1.tar.gz (1.1 MB)
|鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅| 1.1 MB 3.3 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Requirement already satisfied: appdirs in .\python\lib\site-packages (from black) (1.4.4)
Requirement already satisfied: toml>=0.10.1 in .\python\lib\site-packages (from black) (0.10.2)
Requirement already satisfied: pathspec<1,>=0.6 in .\python\lib\site-packages (from black) (0.8.1)
Requirement already satisfied: typed-ast>=1.4.0 in .\python\lib\site-packages (from black) (1.4.1)
Requirement already satisfied: regex>=2020.1.8 in .\python\lib\site-packages (from black) (2020.11.13)
Requirement already satisfied: mypy-extensions>=0.4.3 in .\python\lib\site-packages (from black) (0.4.3)
Requirement already satisfied: click>=7.1.2 in .\python\lib\site-packages (from black) (7.1.2)
Requirement already satisfied: typing-extensions>=3.7.4 in .\python\lib\site-packages (from black) (3.7.4.3)
Building wheels for collected packages: black
Building wheel for black (PEP 517) ... done
Created wheel for black: filename=black-0.0.0-py3-none-any.whl size=124180 sha256=9d02c9cda7598e7caf1f73046db0bb8db2a1baf63bb54ac8c606c7ea8b143f82
Stored in directory: C:\Users\ROBINW~1\AppData\Local\Temp\pip-ephem-wheel-cache-h9rra3jz\wheels\c5\85\79\f3af8daaf8037c0bf14beb3b7a1511a39b6e6902ca2aaf494e
Successfully built black
Installing collected packages: black
WARNING: The scripts black-primer.exe, black.exe and blackd.exe are installed in 'c:\pepys-deploy\pepys-import\python\Scripts' which is not on PATH.
Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed black-0.0.0
Note: the legacy resolver will be going away in the next pip release (Jan 2021 apparently), so it isn't a long-term solution.
robintw
on 30 Nov 2020
Just as an additional piece of information: the install works when when you tell pip to use it's legacy resolver which isn't as strict about metadata (see below). However, you can see that it is definitely confused about versions - it states
Successfully installed black-0.0.0at the end.c:\pepys-deploy\pepys-import>python -m pip install black --no-cache-dir --use-deprecated=legacy-resolver Collecting black Downloading black-20.8b1.tar.gz (1.1 MB) |鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅鈻堚枅| 1.1 MB 3.3 MB/s Installing build dependencies ... done Getting requirements to build wheel ... done Preparing wheel metadata ... done Requirement already satisfied: appdirs in .\python\lib\site-packages (from black) (1.4.4) Requirement already satisfied: toml>=0.10.1 in .\python\lib\site-packages (from black) (0.10.2) Requirement already satisfied: pathspec<1,>=0.6 in .\python\lib\site-packages (from black) (0.8.1) Requirement already satisfied: typed-ast>=1.4.0 in .\python\lib\site-packages (from black) (1.4.1) Requirement already satisfied: regex>=2020.1.8 in .\python\lib\site-packages (from black) (2020.11.13) Requirement already satisfied: mypy-extensions>=0.4.3 in .\python\lib\site-packages (from black) (0.4.3) Requirement already satisfied: click>=7.1.2 in .\python\lib\site-packages (from black) (7.1.2) Requirement already satisfied: typing-extensions>=3.7.4 in .\python\lib\site-packages (from black) (3.7.4.3) Building wheels for collected packages: black Building wheel for black (PEP 517) ... done Created wheel for black: filename=black-0.0.0-py3-none-any.whl size=124180 sha256=9d02c9cda7598e7caf1f73046db0bb8db2a1baf63bb54ac8c606c7ea8b143f82 Stored in directory: C:\Users\ROBINW~1\AppData\Local\Temp\pip-ephem-wheel-cache-h9rra3jz\wheels\c5\85\79\f3af8daaf8037c0bf14beb3b7a1511a39b6e6902ca2aaf494e Successfully built black Installing collected packages: black WARNING: The scripts black-primer.exe, black.exe and blackd.exe are installed in 'c:\pepys-deploy\pepys-import\python\Scripts' which is not on PATH. Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location. Successfully installed black-0.0.0Note: the legacy resolver will be going away in the next pip release (Jan 2021 apparently), so it isn't a long-term solution.
Yup sure thing. I'll fix and push for a release before then. Thanks for raising.
cooperlees
on 30 Nov 2020
I'll also look into this soon. I think this has something to do with setuptools-scm breaking as that's how the version is resolved in our source distributions. Why is it not working as expected? I don't know any answers but I'll do some digging.
ichard26
on 30 Nov 2020
I imagine a partial solution would be to publish a wheel for the latest version - installing the latest version that had a wheel (19.x) worked fine for me. Obviously it'd be good to work out why it's actually happening too though!
robintw
on 30 Nov 2020
We want to do more releases more often so I'm totally +1 for a new release. Need to check that the CHANGELOG is ready though. One thing, I can't do a release. I clearly have some power to push for a release, but no power to create and push one. That power lies with the other collaborators :-)
@cooperlees, what do you think?
ichard26
on 30 Nov 2020
Since @cooperlees asked nicely...
You seem to be hitting https://github.com/pypa/setuptools_scm/issues/386. The main solution is listed in https://github.com/pypa/setuptools_scm/issues/386#issuecomment-585075432. I'm not a 100% sure how you're generating release artifacts, (I don't see your release process documented -- it's usually a good idea to do so, to avoid mistakes!) but it does seem that the sdist doesn't have the metadata that setuptools_scm needs to properly version it, likely due to setuptools_scm not being installed/used in the build environment.
Also, even though this is a pure-Python project, I strongly recommend uploading wheels to PyPI. Most users who use pip won't really download/uses the sdist if there's a wheel, so you'd also buy some time to investigate the underlying issue here by uploading a well-formed wheel. I'll leave it up to the maintainers to decide whether they want the 20.8 beta wheel built (checkout the 20.8 tag and build the wheel with pip wheel .?) or if they wanna put out a completely new release. :)
pradyunsg
on 1 Dec 2020
at first glance i believe a part of the problem is that setup_requires is in setup.cfg but the recent enough setuptools version is not ensured
a second part is that its observable that pip nukes the ability of setuptools_scm to fetch version data when it manages copying things to tmpdirs (and pip does not provide a way to get access to that metadata)
RonnyPfannschmidt
on 5 Dec 2020
Related issues
layoaster
路
3Comments
craigmac
路
3Comments
brettcannon
路
3Comments
JelleZijlstra
路
3Comments
asottile
路
3Comments
Most helpful comment
Just as an additional piece of information: the install works when when you tell pip to use it's legacy resolver which isn't as strict about metadata (see below). However, you can see that it is definitely confused about versions - it states
Successfully installed black-0.0.0at the end.Note: the legacy resolver will be going away in the next pip release (Jan 2021 apparently), so it isn't a long-term solution.