pipx install doesn't prompt for credentials for private vcs url

[alexkillen]

Describe the bug
Regular pipx install command doesn't work for private vcs urls that require authentication. Running pipx install --spec git+https://<some_private_repo>.git <package_name> does not prompt the user for username and password. If the --verbose option is provided, it works fine.

How to reproduce
Run above command using a private vcs url (e.g. github, gitlab, bitbucket) and do not use --verbose.

Expected behavior
Should prompt user for credentials, even when --verbose is not used.

Environment

$ python3 --version
Python 3.6.8
$ pipx --version
0.14.0.0
$ head -n 2 /etc/os-release
NAME="Ubuntu"
VERSION="18.04.3 LTS (Bionic Beaver)"
$ uname -srm
Linux 4.15.0-64-generic x86_64

[cs01]

My guess is that this is because the pipx animation takes over the terminal and hides the prompt for the password. The animation runs until the subprocess command finishes. Does it just animate forever and never finish?

The fix would be to not do the animation. Ways to do this:

• Use verbose flag as you discovered
• Add flag or env var to pipx to suppresses animation
• Never show animation
• Document problem and workaround in cli help text

[alexkillen]

Yes, it animates forever and never completes. As for a solution, a good compromise might be to disable animations when the --spec argument is used. If that's not an option I'd lean towards a --no-anims type flag or env var to disable animations, as using --verbose is not that intuitive, I only discovered that worked while trying to debug the issue.

It would of course be nice if pipx could detect when user prompts occur and drop out of the animation when they do, but that sounds like it would require considerable effort.

btw, pipx is great, the Python ecosystem has needed this for years, too many people try to force pip and pypi into this role and it causes a lot of confusion. Keep up the great work.

[cs01]

@uranusjr how does pipenv solve this?

[alexkillen]

Had a quick check through the pipenv source and it looks like it's using this: https://github.com/sarugaku/vistir#spinner

[cs01]

Thanks!

What happens when installing from a protected source with pipenv? Does it spin indefinitely or does it stop spinning and drop back to the user/password prompt? If it drops back to the password prompt I'll take a look around the visitr source code to see how it does it. I don't want to add it as a dependency because it depends on quite a few things, and I would like to keep pipx as dependency-free as possible.

[cs01]

I wonder if writing to a different stream might fix the issue too. Pipx writes the spinner to stderr. If pip writes to stderr, having pipx do the animation on stdout might fix it.

https://github.com/pipxproject/pipx/blob/master/pipx/animate.py

[uranusjr]

@cs01 Pipenv doesn’t have a solution either; it requires the credentials before installation (using environment variable injection). We have yet to find a good way to interact with pip.

[alexkillen]

When I install from a private VCS using pipenv, it does actually prompt me for username and password, but the spinner still shows and keeps showing wherever the cursor is, leaving behind a trail of individual animation frames of the spinner, so it looks quite odd and isn't really an ideal solution.

I think since usage of a private VCS isn't really going to be widespread, going with the simplistic approach of having a flag to disable animations and a section in the docs about using it with private VCS URLs might be the best "solution" for the moment. Unless you find that changing the animation stream works of course.

[itsayellow]

I just ran into this recently during #302 , when I just deleted one character from my public github repo name, in order to try and force an error. Instead of getting an error that there was no package there, I got a github username prompt next to the animation spinner. It was confusing exactly what was going on.

The only way to get out of this was Ctrl-C. It seems like at least maybe if we have no good way of the user entering username/password we should find a way to disallow it, or to exit early when we see it.

[cs01]

Pipenv doesn’t have a solution either; it requires the credentials before installation (using environment variable injection). We have yet to find a good way to interact with pip.

We could try to use environment variables like pipenv, though @alexkillen's comment indicates pipenv doesn't have behavior any better than pipx, at least in some circumstances.

Another thought is maybe pip detects whether stdout/stderr is a tty and modifies its behavior based on that. Oftentimes programs will have different interactive/non-interactive based on whether the file descriptor is a tty. If we make all of pipx's calls to pip redirect to a file, then pip will not be writing to a tty and maybe it will abort immediately when it reaches a password prompt.

[uranusjr]

For git+ (and other VCS) specifically pip does little than forwarding the URL to the VCS command, so I think the problem is in Git (or even more specifically the credential helper).

For regular HTTP(S) access pip doesn’t do it that well either, see pypa/pip#2920.

[cs01]

It looks like pip's undocumented --no-input flag may do what we're looking for https://github.com/pypa/pip/issues/2920#issuecomment-570124898.