Pipx: Add pyenv compatibility

[Hultner]

I've got a CLI-tool (tmux-res) I built for handling some common tasks in my TMUX-environment which I built a while back. For fun I wanted to try installing it via pipx but it doesn't work.

This is the package:
https://pypi.org/project/tmux-res/

 $ pyenv global 3.7.2
 $ python --version
Python 3.7.2

 $ pipx install tmux-res
  Could not find a version that satisfies the requirement tmux-res (from versions: )
No matching distribution found for tmux-res
'/Users/hultner/.local/pipx/venvs/tmux-res/bin/python -m pip install tmux-res -q' failed

 $ pip install tmux-res
Collecting tmux-res
  Downloading https://files.pythonhosted.org/packages/09/80/a0294c244aebbdc9f13c06b182a7c1857618da20eabd1c88c4f449dbdf09/tmux_res-0.4.8-py3-none-any.whl
Collecting python-dotenv (from tmux-res)
  Downloading https://files.pythonhosted.org/packages/8c/14/501508b016e7b1ad0eb91bba581e66ad9bfc7c66fcacbb580eaf9bc38458/python_dotenv-0.10.1-py2.py3-none-any.whl
Collecting Click (from tmux-res)
  Using cached https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl
Installing collected packages: python-dotenv, Click, tmux-res
Successfully installed Click-7.0 python-dotenv-0.10.1 tmux-res-0.4.8
You are using pip version 18.1, however version 19.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

[cs01]

Thanks for reporting the issue!

I tried and got the same

>> pipx install tmux-res
  Could not find a version that satisfies the requirement tmux-res (from versions: )
No matching distribution found for tmux-res
'/home/csmith/.local/pipx/venvs/tmux-res/bin/python -m pip install tmux-res -q' failed

Since I installed pipx with python3.6, pipx uses that version of python to install other packages by default. I saw that tmux-res requires python3.7. If I specify that version explicitly it works:

>> pipx install tmux-res --python python3.7
  installed package tmux-res 0.4.8, Python 3.7.1
  These binaries are now globally available
    - tmux-res
done! ✨ 🌟 ✨

So I think you have two fixes available: 1: specify the version explicitly when you install, or 2: reinstall pipx with python3.7 so it uses that version by default.

[Hultner]

@cs01 Oh I see, I though it used whatever Python-version I had globally set my pyenv to.
Are there any plans on integrating with pyenv? I'd love to help if you'd point me in the right direction.

[cs01]

I didn't have any plans, but it would be pretty easy to change DEFAULT_PYTHON here.

It could look for an environment variable, something like PIPX_USE_SYSTEM_PYTHON or PIPX_PYENV_PYTHON and use python as the default (which I think would use pyenv's python). If not set, it could keep using sys.executable.

[cs01]

There was actually another issue asking to NOT work with pyenv, which is why pipx uses sys.executable by default now 😄 .

https://github.com/pipxproject/pipx/issues/17

[Hultner]

Having the default behaviour configurable via a environment variable coupled with appropriate documentation sounds like the best solution then from my point of view then.

I could give it a shot if it’s okay.

[Hultner]

And I almost forgot to say thank you I’m your hard working making this took a reality, it does really feel like a better pipsi to me.

[cs01]

Having the default behaviour configurable via a environment variable coupled with appropriate documentation sounds like the best solution then from my point of view then.I could give it a shot if it’s okay.

Sounds good! Thank you.

[Hultner]

Hopefully I'll have some time to give it a try tomorrow night.

[Hultner]

I was a little bit optimistic about my time availability this week, life/family happened. But I'll take a look at it as soon as I have some time to spare, hopefully soon :)

[ABitMoreDepth]

Just bumped into this, is there anything I can do to assist getting this working?

[cs01]

@ABitMoreDepth I am not a pyenv user, but pull requests are welcome if you have a fix.

[robgolding]

I worked around this by launching pipx using the currently-active python (though you must also set PYTHONPATH to ensure it is importable):

PYTHONPATH=$(brew --prefix)/lib/python3.7/site-packages python $(which pipx)

I should note that I did try the suggested modification in this issue (setting DEFAULT_PYTHON = 'python'), but unfortunately the __PYVENV_LAUNCHER__ environment variable causes havoc with the venv package, meaning the virtual environments actually get symlinked to the wrong place (see https://bugs.python.org/issue22490).

I found that I could make things work by deleting that environment variable before calling subprocess.run (via del os.environ['__PYVENV_LAUNCHER__']), but as the technique I described above works I felt it was unnecessary in the end.

[tedmiston]

Is it possible to upgrade the default python used by pipx?

I upgraded to Python 3.8 in pyenv, set that as the global default and when I run python --version get 3.8, but pipx install ... without specifying the --python flag still uses the old version (3.7.4) by default.

I did a pipx reinstall-all python3.8 thinking that might also bump the default, however, future installs after this without a python version flag are still using 3.7.4. I've looked across the docs but didn't see anything about upgrading pipx's default python version.

Is the standard flow to uninstall and reinstall pipx in the new Python user virtual environment to change the default?

Edit: My workaround for now was to just note the packages I had installed, remove pipx, reinstall it in 3.8.0, and run a pipx install for each package. I'd be curious to know if others have a better workflow for this.

[cs01]

pipx is like any other python package with an entry point. If you run which pipx you'll get a path that gets executed. I get

>> which pipx
/home/csmith/.local/bin/pipx

At the top of /home/csmith/.local/bin/pipx is #!/usr/bin/python3.6 since I installed with python3.6 -m pip install --user pipx.

If I change it to #!/usr/bin/python3.7 it will try to use 3.7 and will search the site-packages directory for python3.7, and will fail since it was installed for python3.6.

Installing it with 3.7 and then trying again works fine. And subsequent packages are installed with the newer version since that's what the pipx executable file runs at startup: /usr/bin/python3.7.

> python3.6 -m pip install pipx --user
> pipx install black
> python3.6 -m pip uninstall pipx
> python3.7 -m pip install pipx --user
> pipx install pycowsay
> pipx list
venvs are in /home/csmith/.local/pipx/venvs
apps are exposed on your $PATH at /home/csmith/.local/bin
   package black 19.3b0, Python 3.6.8
    - black
    - blackd
   package pycowsay 0.0.0.1, Python 3.7.3
    - pycowsay

[tedmiston]

Thanks for this.

I see a different shebang for my local setup... not sure if something is weird about my setup or just that I'm using pipx via pyenv (using Python 3.8 if it makes a difference).

$ which pipx
/Users/taylor/.pyenv/shims/pipx
$ head -n 1 $(which pipx)
#!/usr/bin/env bash

Do you happen to know if there is a better (or more automatic) workflow for migrating all of one's packages installed by pipx under e.g., Python 3.7 to another version, e.g., Python 3.8 built-in to pipx?

[itsayellow]

Yes, the ~/.pyenv/shims directory is pyenv's way of handling python versions for apps in there. The "apps" in this directory are actually bash scripts which then dispatch the commands to the proper python version.

I use pyenv, but specifically used "system" as the pyenv version when installing pipx via pip. I think this is why I don't have pyenv create a pipx shim for me, mine is in /usr/local/bin because that's where my "system" python resides (I'm on macos using Homebrew to install python.)

[itsayellow]

Do you happen to know if there is a better (or more automatic) workflow for migrating all of one's packages installed by pipx under e.g., Python 3.7 to another version, e.g., Python 3.8 built-in to pipx?

The python version of your pipx packages is not necessarily the same that pipx uses. It is a separate matter. Use pipx reinstall-all <python_executable_path> to reinstall all of your pacakges installed by pipx and use a different python executable.