Pip-tools: piptools fails with newest click release

[nicoa]

Hey,

not sure if I'm correct here or over at click issues, please let me know if information is missing or this is the wrong place!

Environment Versions

1. docker python:3.8-buster and debian
2. Python version: Python 3.8.10 and Python 3.9.5
3. pip version: 21.1.1
4. pip-tools version: pip-tools-6.1.0

Steps to replicate

console pip install --upgrade pip click pip install pip-tools pip-compile --output-file requirements.txt requirements.in

Expected result

This worked under click 7.1.2, so I'd expect it to work after upgrading as well as the dependency is not pinned.

Actual result

After using newest click (click-8.0.0), the following error was raised:

python-traceback pip 21.1.1 from /usr/local/lib/python3.8/site-packages/pip (python 3.8) Traceback (most recent call last): File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 384, in callback version = metadata.version(package_name) # type: ignore File "/usr/local/lib/python3.8/importlib/metadata.py", line 530, in version return distribution(distribution_name).version File "/usr/local/lib/python3.8/importlib/metadata.py", line 503, in distribution return Distribution.from_name(distribution_name) File "/usr/local/lib/python3.8/importlib/metadata.py", line 177, in from_name raise PackageNotFoundError(name) importlib.metadata.PackageNotFoundError: piptools During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/bin/pip-compile", line 8, in <module> sys.exit(cli()) File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1134, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1058, in main with self.make_context(prog_name, args, **extra) as ctx: File "/usr/local/lib/python3.8/site-packages/click/core.py", line 927, in make_context self.parse_args(ctx, args) File "/usr/local/lib/python3.8/site-packages/click/core.py", line 1376, in parse_args value, args = param.handle_parse_result(ctx, opts, args) File "/usr/local/lib/python3.8/site-packages/click/core.py", line 2352, in handle_parse_result value = self.process_value(ctx, value) File "/usr/local/lib/python3.8/site-packages/click/core.py", line 2314, in process_value value = self.callback(ctx, self, value) File "/usr/local/lib/python3.8/site-packages/click/decorators.py", line 386, in callback raise RuntimeError( RuntimeError: 'piptools' is not installed. Try passing 'package_name' instead.
It seems to be related to click now using importlib instead of pkg_resources: https://github.com/pallets/click/pull/1582 ( see https://click.palletsprojects.com/en/8.0.x/changes/#version-8-0 as well).

Update: I wasn't able to reproduce this under my local python 3.8.2. I was able to reproduce it on another machine (debian ) after installing python 3.8.10 and only on global installation, not when using virtualenvs.

Update 2: As well reproducible on debian machine with 3.9.5 in global/user scope, without virtualenv.

[davidism]

Due to differences between how importlib.metadata works compared to the previous pkg_resources, Click can only detect the version based on the package name. The Python package name must match the installed package name for automatic detection to work. In pip-tools' case, the Python package name is piptools, but the installed name is pip-tools. You'll need to tell the version option this:

@click.version_option(package_name='pip-tools')

[nicoa]

Hey @davidism thanks for the nice explanation, definitely makes sense! I'm still curious why this wasn't reproducible on a fresh virtualenv but only on global scope for me.

If nobody else is faster, I'd try to provide a PR this week to fix this :)

[ehontoria]

I'm also seeing this same issue. Looking forward to the fix.