Originally found while triaging https://github.com/pypa/pip/issues/8174
Environment
Description
On performing an install using a direct URL in a virtualenv with wheel
present, the output of pip freeze
and pip list --format=freeze
differ in the way the package is listed
Expected behavior
pip list --format=freeze
follows the output format style of pip freeze
, and so if the output of pip freeze changes in the case of installations with direct URL, pip list --format=freeze
should follow suit and do the same.
How to Reproduce
pip freeze
and pip list --format=freeze
differOutput
$ python -m venv venv
$ source venv/bin/activate
(venv)$ pip install -U pip
Collecting pip
Using cached https://files.pythonhosted.org/packages/54/2e/df11ea7e23e7e761d484ed3740285a34e38548cf2bad2bed3dd5768ec8b9/pip-20.1-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 19.2.3
Uninstalling pip-19.2.3:
Successfully uninstalled pip-19.2.3
Successfully installed pip-20.1
(venv) $ pip install wheel
Collecting wheel
Using cached wheel-0.34.2-py2.py3-none-any.whl (26 kB)
Installing collected packages: wheel
Successfully installed wheel-0.34.2
(venv) $ pip install git+https://github.com/pypa/twine
Collecting git+https://github.com/pypa/twine
Cloning https://github.com/pypa/twine to /private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pip-req-build-shjhv6hl
Running command git clone -q https://github.com/pypa/twine /private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pip-req-build-shjhv6hl
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting pkginfo>=1.4.2
Using cached pkginfo-1.5.0.1-py2.py3-none-any.whl (25 kB)
Requirement already satisfied: setuptools>=0.7.0 in ./venv/lib/python3.8/site-packages (from twine==3.1.2.dev58+gda8d62f) (41.2.0)
Collecting readme-renderer>=21.0
Using cached readme_renderer-26.0-py2.py3-none-any.whl (15 kB)
Collecting requests-toolbelt!=0.9.0,>=0.8.0
Using cached requests_toolbelt-0.9.1-py2.py3-none-any.whl (54 kB)
Collecting requests>=2.20
Using cached requests-2.23.0-py2.py3-none-any.whl (58 kB)
Collecting keyring>=15.1
Using cached keyring-21.2.0-py3-none-any.whl (30 kB)
Collecting tqdm>=4.14
Using cached tqdm-4.45.0-py2.py3-none-any.whl (60 kB)
Collecting bleach>=2.1.0
Downloading bleach-3.1.5-py2.py3-none-any.whl (151 kB)
|ββββββββββββββββββββββββββββββββ| 151 kB 252 kB/s
Collecting docutils>=0.13.1
Using cached docutils-0.16-py2.py3-none-any.whl (548 kB)
Collecting six
Using cached six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting Pygments>=2.5.1
Using cached Pygments-2.6.1-py3-none-any.whl (914 kB)
Collecting chardet<4,>=3.0.2
Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting certifi>=2017.4.17
Using cached certifi-2020.4.5.1-py2.py3-none-any.whl (157 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
Collecting idna<3,>=2.5
Using cached idna-2.9-py2.py3-none-any.whl (58 kB)
Collecting packaging
Using cached packaging-20.3-py2.py3-none-any.whl (37 kB)
Collecting webencodings
Using cached webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Collecting pyparsing>=2.0.2
Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Building wheels for collected packages: twine
Building wheel for twine (PEP 517) ... done
Created wheel for twine: filename=twine-3.1.2.dev58+gda8d62f-py3-none-any.whl size=36764 sha256=4d4a011ba94cc61d89b661977d66816de6b2dd9c23c189a4a77a68eda6997476
Stored in directory: /private/var/folders/xg/blp845_s0xn093dyrtgy936h0000gp/T/pip-ephem-wheel-cache-_wbhcsnm/wheels/73/dd/fb/3039711df8645ab0ccc3e7d5f834289bf9ee7569c222c1fbc4
Successfully built twine
Installing collected packages: pkginfo, six, pyparsing, packaging, webencodings, bleach, docutils, Pygments, readme-renderer, chardet, certifi, urllib3, idna, requests, requests-toolbelt, keyring, tqdm, twine
Successfully installed Pygments-2.6.1 bleach-3.1.5 certifi-2020.4.5.1 chardet-3.0.4 docutils-0.16 idna-2.9 keyring-21.2.0 packaging-20.3 pkginfo-1.5.0.1 pyparsing-2.4.7 readme-renderer-26.0 requests-2.23.0 requests-toolbelt-0.9.1 six-1.14.0 tqdm-4.45.0 twine-3.1.2.dev58+gda8d62f urllib3-1.25.9 webencodings-0.5.1
(venv) $ pip freeze
....
twine @ git+https://github.com/pypa/twine@da8d62f6e8878900267a8499d4e3cfe49ac11002
....
(venv) $ pip list --format=freeze
....
twine==3.1.2.dev58+gda8d62f
....
pip list --format=freeze
should follow suit and do the same.
@deveshks I'm personally not entirely sure both should do the same, or if the difference should be documented (and tested).
Maybe they should do different things, given that thereβs indeed people wanting the old behaviour (i.e. #8174). But it is so entirely not obvious which one should do which. Maybe we should get the new inspect
command going soon to replace them both. Is there a writeup for that somewhere @pradyunsg?
Somewhat related to #7157.
And in this context, where pip freeze
would be an alias to pip list --format=freeze
(or pip inspect --format=freeze
?), this would be a bug :)
Ideally we should be able to output both format. Maybe a --format=freeze
& --format=freeze-version
?
I think if we want to start looking for a pip inspect
to replace list
and freeze
, now along with supported options to keep relevant outputs from both, that should be able to make the 20.2 train in July if we come up with a documentation, and an implementation in the next 3 months?
After that, we can decide on the deprecation cycle to remove list
and freeze
.
Do we want to switch to https://github.com/pypa/pip/issues/7157 to start discussing on that, and include the issue here as part of that discussion? Or do we want to fix this in the next release, along with the inspect
command?
Is there any way to prevent pip freeze
from writing the direct path to the local wheel used to install from? Before Pip 20.1 and PEP 610 I could just pip freeze > requirements.txt
after upgrading packages and not worry which directory I happen to be installing wheels from. This helps decouple local install paths from generic version requirements.
pip list --format=freeze
seems to be one way that works to avoid the above issue.
pip wouldnβt record the packageβs source URL if you donβt install the package by its path/URL:
pip install --no-index --find-links directory/containing/wheel package-name
@uranusjr that works if you only have one or two packages to install or already have a requirements.txt ready.
What about installing 30 new packages from wheels? I can easily script it to βββpip install βno-index -f . [List of all wheels names in current directory]βββ. However, with new pip, freezing such an install brings the complete wheel path into the requirements.txt. :(
To follow your suggestion I would have to somehow strip out all the wheel file names down to just package names.
Is there another easier way to install all wheels In a directory regardless what wheels are in there?
Using bash, this is an easy way to get the package names:
pip install --no-index --find-links wheelsdir $(ls wheelsdir/*.whl | cut -d '-' -f 1)
At this stage, pip list --format=freeze
also does what you want, as you noted. It is further being discussed in #8176. Ow, sorry, this is #8176.
Iβm doing this in Windows 10, but the install is a python script. Iβll see about just stripping out the package names.
Yes, pip list --format=freeze
covers this corner case for now. It would be good if this option remains if you want to build a pure requirements.txt regardless of installation from wheel or pypi.