Pipx: issue with pipx dependency conflicting with package dependency, Click

Created on 9 Oct 2019  Â·  11Comments  Â·  Source: pipxproject/pipx

I'm running into an issue with the clokta package. It depends on Click. It seems like it is pipx thinks it is already installed because Click is a dependency of pipx itself.

Requirement already satisfied: click in /usr/local/Cellar/pipx/0.14.0.0/libexec/lib/python3.7/site-packages (from clokta) (7.0)

╰─ pipx install clokta --verbose
pipx > (run_pipx_command:134): Virtual Environment location is /Users/johria/.local/pipx/venvs/clokta
pipx > (run:97): running /usr/local/opt/python/bin/python3.7 -m venv --without-pip /Users/johria/.local/pipx/venvs/clokta
pipx > (run:97): running /Users/johria/.local/pipx/venvs/clokta/bin/python -m pip install clokta
Collecting clokta
  Using cached https://files.pythonhosted.org/packages/7c/de/02c8f5a51ebacea04e52745852a3dc691d40e78d0fdf0be9f53ea2855561/clokta-4.1.0-py3-none-any.whl
Collecting enum-compat (from clokta)
Collecting boto3 (from clokta)
  Using cached https://files.pythonhosted.org/packages/42/b5/03da45c451b4e7d31bc42e3b844cbb2e9fc6e8757705504d828a7295b27e/boto3-1.9.245-py2.py3-none-any.whl
Collecting beautifulsoup4 (from clokta)
  Using cached https://files.pythonhosted.org/packages/3b/c8/a55eb6ea11cd7e5ac4bacdf92bac4693b90d3ba79268be16527555e186f0/beautifulsoup4-4.8.1-py3-none-any.whl
Collecting keyring (from clokta)
  Using cached https://files.pythonhosted.org/packages/b1/08/ad1ae7262c8146bee3be360cc766d0261037a90b44872b080a53aaed4e84/keyring-19.2.0-py2.py3-none-any.whl
Collecting requests (from clokta)
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting configparser (from clokta)
  Using cached https://files.pythonhosted.org/packages/7a/2a/95ed0501cf5d8709490b1d3a3f9b5cf340da6c433f896bbe9ce08dbe6785/configparser-4.0.2-py2.py3-none-any.whl
Requirement already satisfied: click in /usr/local/Cellar/pipx/0.14.0.0/libexec/lib/python3.7/site-packages (from clokta) (7.0)
Collecting botocore<1.13.0,>=1.12.245 (from boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/0e/84/b13b233a344de26796b7b457571c16f2b3d1fab8fb48931208d650cc4d97/botocore-1.12.245-py2.py3-none-any.whl
Collecting s3transfer<0.3.0,>=0.2.0 (from boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl
Collecting jmespath<1.0.0,>=0.7.1 (from boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Collecting soupsieve>=1.2 (from beautifulsoup4->clokta)
  Using cached https://files.pythonhosted.org/packages/5d/42/d821581cf568e9b7dfc5b415aa61952b0f5e3dede4f3cbd650e3a1082992/soupsieve-1.9.4-py2.py3-none-any.whl
Collecting entrypoints (from keyring->clokta)
  Using cached https://files.pythonhosted.org/packages/ac/c6/44694103f8c221443ee6b0041f69e2740d89a25641e62fb4f2ee568f2f9c/entrypoints-0.3-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests->clokta)
  Using cached https://files.pythonhosted.org/packages/e0/da/55f51ea951e1b7c63a579c09dd7db825bb730ec1fe9c0180fc77bfb31448/urllib3-1.25.6-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5 (from requests->clokta)
  Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests->clokta)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests->clokta)
  Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting python-dateutil<3.0.0,>=2.1; python_version >= "2.7" (from botocore<1.13.0,>=1.12.245->boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting docutils<0.16,>=0.10 (from botocore<1.13.0,>=1.12.245->boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/22/cd/a6aa959dca619918ccb55023b4cb151949c64d4d5d55b3f4ffd7eee0c6e8/docutils-0.15.2-py3-none-any.whl
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1; python_version >= "2.7"->botocore<1.13.0,>=1.12.245->boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Installing collected packages: enum-compat, six, python-dateutil, jmespath, urllib3, docutils, botocore, s3transfer, boto3, soupsieve, beautifulsoup4, entrypoints, keyring, idna, chardet, certifi, requests, configparser, clokta
Successfully installed beautifulsoup4-4.8.1 boto3-1.9.245 botocore-1.12.245 certifi-2019.9.11 chardet-3.0.4 clokta-4.1.0 configparser-4.0.2 docutils-0.15.2 entrypoints-0.3 enum-compat-0.0.2 idna-2.8 jmespath-0.9.4 keyring-19.2.0 python-dateutil-2.8.0 requests-2.22.0 s3transfer-0.2.1 six-1.12.0 soupsieve-1.9.4 urllib3-1.25.6
pipx > (_symlink_package_apps:551): Same path /Users/johria/.local/bin/clokta and /Users/johria/.local/pipx/venvs/clokta/bin/clokta
  installed package clokta 4.1.0, Python 3.7.4
  These apps are now globally available
    - clokta
done! ✨ 🌟 ✨

This is installed via the latest homebrew release.

$ clokta --version
clokta, version 4.1.0
help wanted
Source
picture AlJohri
👍2

Most helpful comment

It looks fine to me. If you make the change in a PR and the unit tests still pass then I will merge it.

picture cs01 on 15 Oct 2019
👍2

All 11 comments

I got around the issue temporarily by force installing it into the virtualenv: $HOME/.local/pipx/venvs/clokta/bin/python -m pip install click.

AlJohri picture AlJohri on 9 Oct 2019

I just ran pipx install clokta --verbose but did not get that warning. Maybe it has something to do with pipx being installed with homebrew. Have you tried installing pipx with pip, then installing clokta?

cs01 picture cs01 on 9 Oct 2019

@cs01 yes, it does not occur with pipx installed via pip. however, I'm not sure its entirely a homebrew issue

going back to the homebrew installed version, here are some debug details:

$ which pipx
/usr/local/bin/pipx

$ pipx --help
...
Virtual Environment location is /Users/johria/.local/pipx/venvs.
Symlinks to apps are placed in /Users/johria/.local/bin.
These locations can be overridden with the environment variables
PIPX_HOME and PIPX_BIN_DIR, respectively. (Virtual Environments will
be installed to $PIPX_HOME/venvs)
...

$ echo $PIPX_HOME # blank
$ echo $PIPX_BIN_DIR # blank

$ pipx list
venvs are in /Users/johria/.local/pipx/venvs
apps are exposed on your $PATH at /Users/johria/.local/bin
   package envdir 1.0.1, Python 3.7.4
    - envdir
    - envshell
   package homebrew-pypi-poet 0.10.0, Python 3.7.4
    - poet
    - poet_lint
   package nbdime 1.1.0, Python 3.7.4
    - git-nbdiffdriver
    - git-nbdifftool
    - git-nbmergedriver
    - git-nbmergetool
    - hg-nbdiff
    - hg-nbdiffweb
    - hg-nbmerge
    - hg-nbmergeweb
    - nbdiff
    - nbdiff-web
    - nbdime
    - nbmerge
    - nbmerge-web
    - nbshow
   package python-language-server 0.28.3, Python 3.7.4
    - pyls

$ pipx install clokta --verbose
pipx > (run_pipx_command:134): Virtual Environment location is /Users/johria/.local/pipx/venvs/clokta
pipx > (run:97): running /usr/local/opt/python/bin/python3.7 -m venv --without-pip /Users/johria/.local/pipx/venvs/clokta
pipx > (run:97): running /Users/johria/.local/pipx/venvs/clokta/bin/python -m pip install clokta
Collecting clokta
  Using cached https://files.pythonhosted.org/packages/7c/de/02c8f5a51ebacea04e52745852a3dc691d40e78d0fdf0be9f53ea2855561/clokta-4.1.0-py3-none-any.whl
Collecting configparser (from clokta)
  Using cached https://files.pythonhosted.org/packages/7a/2a/95ed0501cf5d8709490b1d3a3f9b5cf340da6c433f896bbe9ce08dbe6785/configparser-4.0.2-py2.py3-none-any.whl
Collecting keyring (from clokta)
  Using cached https://files.pythonhosted.org/packages/b1/08/ad1ae7262c8146bee3be360cc766d0261037a90b44872b080a53aaed4e84/keyring-19.2.0-py2.py3-none-any.whl
Collecting requests (from clokta)
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting enum-compat (from clokta)
Collecting beautifulsoup4 (from clokta)
  Using cached https://files.pythonhosted.org/packages/3b/c8/a55eb6ea11cd7e5ac4bacdf92bac4693b90d3ba79268be16527555e186f0/beautifulsoup4-4.8.1-py3-none-any.whl
Requirement already satisfied: click in /usr/local/Cellar/pipx/0.14.0.0/libexec/lib/python3.7/site-packages (from clokta) (7.0)
Collecting boto3 (from clokta)
  Using cached https://files.pythonhosted.org/packages/ef/f3/4ef8264ed1cf4628513de5537464da1cf60b4f8e7551c4f2611e1e3cb4bd/boto3-1.9.247-py2.py3-none-any.whl
Collecting entrypoints (from keyring->clokta)
  Using cached https://files.pythonhosted.org/packages/ac/c6/44694103f8c221443ee6b0041f69e2740d89a25641e62fb4f2ee568f2f9c/entrypoints-0.3-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests->clokta)
  Using cached https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests->clokta)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5 (from requests->clokta)
  Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests->clokta)
  Using cached https://files.pythonhosted.org/packages/e0/da/55f51ea951e1b7c63a579c09dd7db825bb730ec1fe9c0180fc77bfb31448/urllib3-1.25.6-py2.py3-none-any.whl
Collecting soupsieve>=1.2 (from beautifulsoup4->clokta)
  Using cached https://files.pythonhosted.org/packages/5d/42/d821581cf568e9b7dfc5b415aa61952b0f5e3dede4f3cbd650e3a1082992/soupsieve-1.9.4-py2.py3-none-any.whl
Collecting s3transfer<0.3.0,>=0.2.0 (from boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl
Collecting jmespath<1.0.0,>=0.7.1 (from boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Collecting botocore<1.13.0,>=1.12.247 (from boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/86/b2/ce485bd915a1570911765e28d3cca075ff8a920eec11f7a7d0c9725fb730/botocore-1.12.247-py2.py3-none-any.whl
Collecting python-dateutil<3.0.0,>=2.1; python_version >= "2.7" (from botocore<1.13.0,>=1.12.247->boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting docutils<0.16,>=0.10 (from botocore<1.13.0,>=1.12.247->boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/22/cd/a6aa959dca619918ccb55023b4cb151949c64d4d5d55b3f4ffd7eee0c6e8/docutils-0.15.2-py3-none-any.whl
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1; python_version >= "2.7"->botocore<1.13.0,>=1.12.247->boto3->clokta)
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Installing collected packages: configparser, entrypoints, keyring, certifi, chardet, idna, urllib3, requests, enum-compat, soupsieve, beautifulsoup4, six, python-dateutil, docutils, jmespath, botocore, s3transfer, boto3, clokta
Successfully installed beautifulsoup4-4.8.1 boto3-1.9.247 botocore-1.12.247 certifi-2019.9.11 chardet-3.0.4 clokta-4.1.0 configparser-4.0.2 docutils-0.15.2 entrypoints-0.3 enum-compat-0.0.2 idna-2.8 jmespath-0.9.4 keyring-19.2.0 python-dateutil-2.8.0 requests-2.22.0 s3transfer-0.2.1 six-1.12.0 soupsieve-1.9.4 urllib3-1.25.6
pipx > (_symlink_package_apps:551): Same path /Users/johria/.local/bin/clokta and /Users/johria/.local/pipx/venvs/clokta/bin/clokta
  installed package clokta 4.1.0, Python 3.7.4
  These apps are now globally available
    - clokta
done! ✨ 🌟 ✨

As you can see, pipx is running /Users/johria/.local/pipx/venvs/clokta/bin/python -m pip install clokta.

If I manually run, /Users/johria/.local/pipx/venvs/clokta/bin/python -m pip install clokta, I get the following output:

Requirement already satisfied: clokta in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (4.1.0)
Collecting click (from clokta)
  Using cached https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl
Requirement already satisfied: requests in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from clokta) (2.22.0)
Requirement already satisfied: boto3 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from clokta) (1.9.247)
Requirement already satisfied: enum-compat in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from clokta) (0.0.2)
Requirement already satisfied: keyring in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from clokta) (19.2.0)
Requirement already satisfied: beautifulsoup4 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from clokta) (4.8.1)
Requirement already satisfied: configparser in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from clokta) (4.0.2)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from requests->clokta) (1.25.6)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from requests->clokta) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from requests->clokta) (2019.9.11)
Requirement already satisfied: idna<2.9,>=2.5 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from requests->clokta) (2.8)
Requirement already satisfied: s3transfer<0.3.0,>=0.2.0 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from boto3->clokta) (0.2.1)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from boto3->clokta) (0.9.4)
Requirement already satisfied: botocore<1.13.0,>=1.12.247 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from boto3->clokta) (1.12.247)
Requirement already satisfied: entrypoints in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from keyring->clokta) (0.3)
Requirement already satisfied: soupsieve>=1.2 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from beautifulsoup4->clokta) (1.9.4)
Requirement already satisfied: python-dateutil<3.0.0,>=2.1; python_version >= "2.7" in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from botocore<1.13.0,>=1.12.247->boto3->clokta) (2.8.0)
Requirement already satisfied: docutils<0.16,>=0.10 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from botocore<1.13.0,>=1.12.247->boto3->clokta) (0.15.2)
Requirement already satisfied: six>=1.5 in ./.local/pipx/venvs/clokta/lib/python3.7/site-packages (from python-dateutil<3.0.0,>=2.1; python_version >= "2.7"->botocore<1.13.0,>=1.12.247->boto3->clokta) (1.12.0)
Installing collected packages: click
Successfully installed click-7.0

As you can see, manually running /Users/johria/.local/pipx/venvs/clokta/bin/python -m pip install clokta causes Click to get installed.

I think this is a pipx issue, not a homebrew issue @cs01

AlJohri picture AlJohri on 11 Oct 2019
👍1

@cs01, I figured it out.

The way the homebrew formula works is by manipulating the PYTHONPATH: https://github.com/Homebrew/homebrew-core/blob/master/Formula/pipx.rb#L47

Homebrew python packages can work in two ways: 1) using a proper virtualenv and 2) using a global vanilla python where the PYTHONPATH variable points it to the packages it needs. because of the nature of pipx, we use the second

I believe the correct solution will be to explicitly unset the PYTHONPATH within pipx

AlJohri picture AlJohri on 11 Oct 2019
👍1

as a quick proof of concept, this patch fixes the issue:

diff --git a/pipx/util.py b/pipx/util.py
index 94732d8..ba27eee 100644
--- a/pipx/util.py
+++ b/pipx/util.py
@@ -93,11 +93,12 @@ def get_site_packages(python: Path) -> Path:
 def run(cmd: Sequence[Union[str, Path]], check=True) -> int:
     """Run arbitrary command as subprocess"""

+    env = {k:v for k,v in os.environ.items() if k != 'PYTHONPATH'}
     cmd_str = " ".join(str(c) for c in cmd)
     logging.info(f"running {cmd_str}")
     # windows cannot take Path objects, only strings
     cmd_str_list = [str(c) for c in cmd]
-    returncode = subprocess.run(cmd_str_list).returncode
+    returncode = subprocess.run(cmd_str_list, env=env).returncode
     if check and returncode:
         raise PipxError(f"{cmd_str!r} failed")
     return returncode
AlJohri picture AlJohri on 11 Oct 2019
👍1

Same issue here:

➜ [tmuxp4] ~ pipx runpip pingtop install click
Requirement already satisfied: click in /usr/local/Cellar/pipx/0.14.0.0/libexec/lib/python3.7/site-packages (7.0)

Pipx believes click is already installed, but it's pipx's click, not my tools' click.

INFO: my pipx is from brew.

➜ [tmuxp4] ~ brew info pipx
pipx: stable 0.14.0.0 (bottled)
Execute binaries from Python packages in isolated environments
https://github.com/pipxproject/pipx
/usr/local/Cellar/pipx/0.14.0.0 (107 files, 674.8KB) *
  Poured from bottle on 2019-10-15 at 10:38:23
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/pipx.rb
==> Dependencies
Required: python ✔
==> Analytics
install: 277 (30 days), 541 (90 days), 801 (365 days)
install_on_request: 273 (30 days), 537 (90 days), 796 (365 days)
build_error: 0 (30 days)
laixintao picture laixintao on 15 Oct 2019

@AlJohri any quickfix?

laixintao picture laixintao on 15 Oct 2019

@cs01 what do you think of my proposed fix?

AlJohri picture AlJohri on 15 Oct 2019

It looks fine to me. If you make the change in a PR and the unit tests still pass then I will merge it.

cs01 picture cs01 on 15 Oct 2019
👍2

ping @AlJohri ... your proposed change passes the existing unittests ... submit a PR?

It would be great if we could also have a test which demonstrates the problem and shows that it is fixed.

jayvdb picture jayvdb on 23 Oct 2019

Will do

On Wed, Oct 23, 2019 at 2:14 AM John Vandenberg notifications@github.com
wrote:

ping @AlJohri https://github.com/AlJohri ... your proposed change
passes the existing unittests ... submit a PR?

It would be great if we could also have a test which demonstrates the
problem and shows that it is fixed.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pipxproject/pipx/issues/233?email_source=notifications&email_token=AAVJFTA7SFLTKLCUWPLO4HLQP7TUFA5CNFSM4I7A5EPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECAGJ5Q#issuecomment-545285366,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAVJFTFFPHATENHMOATTMEDQP7TUFANCNFSM4I7A5EPA
.

>

  • Al
AlJohri picture AlJohri on 23 Oct 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

petobens picture petobens  Â·  8Comments
cs01 picture cs01  Â·  4Comments
pfmoore picture pfmoore  Â·  6Comments
halloleo picture halloleo  Â·  4Comments
xsc27 picture xsc27  Â·  7Comments