Pipx: Feature request: add skip argument to upgrade-all command

[petobens]

Hi! I've recently discovered that running pipx upgrade on a particular package breaks the symlink. To reproduce:

1. Install unimatrix with

pipx install unimatrix --spec git+https://github.com/will8211/unimatrix --verbose

1. Check that unimatrix binary is correctly symlinked (for instance with which unimatrix)
2. Run pipx upgrade unimatrix -> for some reason pipx this upgrades unimatrix to 0.1 which isn't a tagged release
3. Now try running unimatrix or which unimatrix: it doesn't work since the symlink seems to be broken.

Please let me know if you can reproduce. Thanks!

[cs01]

Haven’t tried to reproduce yet as I am typing this from my phone, but when pipx upgrades a package it uses PyPI by default. Have you tried passing the —spec argument to the upgrade command?

[petobens]

Yep, it stills upgrades it:

pipx upgrade unimatrix --spec unimatrix
upgraded package unimatrix from 0.1.dev66+g75f7402 to 0.1 (location: /home/pedro/.local/pipx/venvs/venvs/unimatrix)

I believe the symlink breaks because if I run

pipx install unimatrix

I get

Installing collected packages: unimatrix
  Running setup.py install for unimatrix ... done
Successfully installed unimatrix-0.1
pipx (rmdir:25): removing directory /home/pedro/.local/pipx/venvs/venvs/unimatrix
No binaries associated with package unimatrix.

So I guess the problem is that pipx is upgrading a package that it should be upgrading?
Also does the upgrade-all command respect the --spec argument?
Finally maybe you are willing to introduce a new freeze or ignore argument to upgrade-all that signals which package not to upgrade?

Thanks for the quick reply btw.

[cs01]

Can you run the commands with the verbose flag and share the output? I think that might help me.

The spec argument can’t be passed to upgrade all. But the idea of adding an ignore flag is really good! I’ll add that.

Pipx just runs “pip install —upgrade” under the hood. Does that command work for you?

[petobens]

> pipx upgrade unimatrix --verbose --spec unimatrix
pipx (run_pipx_command:125): Virtual Environment location is /home/pedro/.local/pipx/venvs/venvs/unimatrix
pipx (_run:124): running /home/pedro/.local/pipx/venvs/venvs/unimatrix/bin/python -m pip install --upgrade pip
Requirement already up-to-date: pip in /home/pedro/.local/pipx/venvs/venvs/unimatrix/lib/python3.7/site-packages (19.0.3)
pipx (_run:124): running /home/pedro/.local/pipx/venvs/venvs/unimatrix/bin/python -m pip install --upgrade unimatrix
Collecting unimatrix
  Using cached https://files.pythonhosted.org/packages/f5/85/4d2fad1fbfb670d9d292c991bc8a4625d4d8966a653c6cada72454811a6d/unimatrix-0.1.tar.gz
Installing collected packages: unimatrix
  Found existing installation: unimatrix 0.1.dev66+g75f7402
    Uninstalling unimatrix-0.1.dev66+g75f7402:
      Successfully uninstalled unimatrix-0.1.dev66+g75f7402
  Running setup.py install for unimatrix ... done
Successfully installed unimatrix-0.1
upgraded package unimatrix from 0.1.dev66+g75f7402 to 0.1 (location: /home/pedro/.local/pipx/venvs/venvs/unimatrix)

But the idea of adding an ignore flag is really good!

Glad you liked it! That would solve my problem (which seems to be related to the package itself right?)

[cs01]

It seems related to the package itself. Running “pip install -U” on the package would test if pip can’t do it. If pip can’t do it then it’s not a bug in pipx. If it can, then it is a bug in pipx.

Also I noticed in your above command that you didn’t pass the spec argument to the upgrade command. Have you tried that?

[petobens]

Also I noticed in your above command that you didn’t pass the spec argument to the upgrade command.

Fixed in my previous comment; still the same output

Running “pip install -U” on the package would test if pip can’t do it. If pip can’t do it then it’s not a bug in pipx. If it can, then it is a bug in pipx.

Running ./pip install -U unimatrix from within the venv also updates the package so it seems to be a problem with pip then which believe the 0.1 to supersede the 0.1.dev version.

Anyways I can leave the issue open and change the title if you are willing to add the ignore flag to the upgrade-all command

[cs01]

Yup sounds good. I think “skip” might be better than “ignore” since ignore makes me think of patters or regular expressions.

But yeah this ticket can serve to implement that idea.

[petobens]

Great. I''ll change the title accordingly. One nit though: after upgrading with pip the unimatrix binary still works whereas when I upgrade with pipx the symlink (and hence the binary) is broken. That does seem a problem with pipx AFAICT

[cs01]

It probably has something to do with the line “No binaries associated with package unimatrix." I will have to try to reproduce it to find out.

[petobens]

Ok. Thanks!

[cs01]

I tried to reproduce, but nothing broke for me

>> pipx install unimatrix --spec git+https://github.com/will8211/unimatrix.git
installed package unimatrix 0.1.dev66+g75f7402, Python 3.6.7
  These binaries are now globally available
    - unimatrix
done! ✨ 🌟 ✨
>> pipx upgrade unimatrix --spec git+https://github.com/will8211/unimatrix.git
unimatrix is already at latest version 0.1.dev66+g75f7402 (location: /home/csmith/.local/pipx/venvs/unimatrix)

and unimatrix still works for me (super cool, btw!)

[petobens]

Oh ok you need to pass the github repo; I was doing pipx upgrade unimatrix --verbose --spec unimatrix.
However my problem is that I frequently use upgrade-all which AFAICT doesn't honor the spec argument .
So in the end I guess the solution is to implement the skip argument? I'll change the title of this issue to reflect that.

[cs01]

So in the end I guess the solution is to implement the skip argument? I'll change the title of this issue to reflect that.

Yup that is the fix. (An alternate fix is to store a mapping of --spec to pipx-installed packages, but I am hesitant to add that)

[cs01]

This has been added to pipx v0.13:

>> pipx upgrade-all -h
usage: pipx upgrade-all [-h] [--include-deps] [--system-site-packages]
                        [--index-url INDEX_URL] [--editable]
                        [--pip-args PIP_ARGS] [--skip SKIP [SKIP ...]]
                        [--verbose]

Upgrades all packages within their virtual environments by running 'pip
install --upgrade PACKAGE'

optional arguments:
  -h, --help            show this help message and exit
  --include-deps        Include binaries of dependent packages
  --system-site-packages
                        Give the virtual environment access to the system
                        site-packages dir.
  --index-url INDEX_URL, -i INDEX_URL
                        Base URL of Python Package Index
  --editable, -e        Install a project in editable mode
  --pip-args PIP_ARGS   Arbitrary pip arguments to pass directly to pip
                        install/upgrade commands
  --skip SKIP [SKIP ...]
                        skip these packages
  --verbose

[petobens]

Thank you! However I've detected a minor problem, if i run pipx upgrade-all without the --skip flag I get:

Traceback (most recent call last):
  File "/home/pedro/.local/bin/pipx", line 10, in <module>
    sys.exit(cli())
  File "/home/pedro/.local/lib/python3.7/site-packages/pipx/main.py", line 525, in cli
    exit(run_pipx_command(parsed_pipx_args, binary_args))
  File "/home/pedro/.local/lib/python3.7/site-packages/pipx/main.py", line 198, in run_pipx_command
    skip=args.skip,
  File "/home/pedro/.local/lib/python3.7/site-packages/pipx/commands.py", line 261, in upgrade_all
    if package in skip:
TypeError: argument of type 'NoneType' is not iterable

[cs01]

Ah thanks for reporting! Fixed in 0.13.0.1.

[petobens]

Thanks.