Pipx: issues with userpath dependency

[AlJohri]

I have some issues with the direct and hard dependency on userpath.

The main issue is the dependency on a separate top level _python_ CLI tool: I use pipx to wrap ALL python CLI tools (that are not available in a package manager, i.e. homebrew). The setup instructions for pipx now require a separate python CLI tool which I ideally would have installed through pipx itself (pipx install userpath).

When installing via homebrew, only pipx binaries are added to the PATH- cli tools from dependencies are not added. Since I don't install any python packages globally (pip freeze is empty), I can't run python3 -m userpath ....

I recommend either:

1. make userpath a subcommand of pipx similar to ensurepath OR

2. taking the approach of pyenv and several other libraries out there of telling users to pipe the command into their shell profile:

   echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bash_profile
   

   • Zsh note: Modify your ~/.zshenv file instead of ~/.bash_profile.
   • Ubuntu and Fedora note: Modify your ~/.bashrc file instead of ~/.bash_profile

[AlJohri]

cc @ofek

[cs01]

@AlJohri pipx lists userpath as a dependency in its setup.py file. Does homebrew call pip or somehow install the dependencies listed in setup.py? If so, python3 -m userpath ... should work.

@jaraco I believe you had thoughts on this?

[jaraco]

1. taking the approach of pyenv and several other libraries out there of telling users to pipe the command into their shell profile

This approach requires the documentation to either take into account all possible environments (Windows, bash, other shells) or require the user to translate the instructions to those environments. I'd really like to see a tool that handles all these cases. userpath does that.

1. make userpath a subcommand of pipx similar to ensurepath

My feeling was that by making it a subcommand, it requires pipx to supply an interface, essentially duplicating the interface that userpath already presents, and maintaining a coupling between the two libraries.

Instead, by merely requiring the userpath dependency, there's a nice separation of concerns. userpath focuses on its specialty and pipx focuses on its own and defers to userpath when path manipulation is needed.

When installing via homebrew, only pipx binaries are added to the PATH.

Perhaps that's the issue. If pipx, by declaring a dependency on userpath, is in fact expecting both pipx and userpath to be present in the same PATH. I'm surprised that the Homebrew recipe doesn't cause the CLI commands to be installed. Perhaps the Homebrew recipe can amended to support that.

The setup instructions for pipx now require a separate python CLI tool which I ideally would have installed through pipx itself. ... I don't install any python packages globally.

I do agree - if you wish not to have Python packages globally installed, you should not have to. But I do think that it's not unreasonable for a package (like pipx) to have a dependency on sister packages to supply complementary functionality.

---

My recommendation would be to investigate whether Homebrew could honor pipx's new expectation that userpath will be present alongside pipx.

[jaraco]

@AlJohri I now see that you are the contributor of the Homebrew formula. I'm going to peek at your contribution there, but I'm also eager to hear your response to my thoughts above.

[jaraco]

Perhaps the Homebrew formula _should_ install pipx without userpath and ensure that the path is installed in the user's shell... our install userpath (maybe temporarily) to ensure the path is configured with some preferred default.

[ofek]

Please let me know if I need to do anything for userpath. I love contributions, too 🙂

[AlJohri]

@cs01 yes, python3 -m userpath would work if pipx was installed into the global python. pipx dependencies are installed and isolated (via PYTHONPATH manipulation) to just pipx so the enduser does not have access to them. Happy to go into more details on the mechanics if you like.

@jaraco I can of course change the homebrew formula to install two top-level tools- userpath and pipx but I don't think homebrew maintainers would be too keen on it. They are in fact two separate tools. The proper homebrew way of doing this would be to depends_on "userpath" as one might depend on any other CLI tool such as wget or jq, etc. However, userpath is neither a homebrew package nor does it meet the popularity threshold to turn it into a homebrew package at this time (sorry @ofek!)

I think using args, unknownargs = parser.parse_known_args() of argparse, its fairly simple to pass through a complete command to a subtool.

Personally, I agree that this isn't great separation of concerns...BUT 1) pipx is useless if the PATH is set incorrectly, 2) there is no universally agreed upon and popular enough solution for userpath manipulation (@ofek hopefully yours will be soon!), and 3) having dozens of lines of documentation accounting for bash, zsh, fish, and you name it sucks- so I think wrapping the command is the best option.

let me know if you think of another solution!

[AlJohri]

okay after attempting to implement that, ignore that parse_known_args comment. the implementation is ugly and the UX sucks. you'll have to just mimic the interface or do an ensurepath style abstraction.

[jaraco]

Hi @AlJohri. Thanks for giving that approach a go.

Thinking about the UX, pipx ensurepath seems unnecessarily entangled, when another process exists to do that work.

I don't think homebrew maintainers would be too keen on the formula installing two top-level tools.

Perhaps not, but it seems they have two contradictory constraints. If we assume for a moment that the current implementation of two separate tools in pipx is preferable, then ideally there should be a solution in Homebrew. Either:

• userpath is not popular enough to be included in Homebrew, in which case it should be vendored (supplied as its own command) in the packages that require it (currently pipx), or
• because pipx requires it, it now is by definition _at least_ as popular as pipx, and so a dependency formula is justified.

Even if we elect neither of those options, I believe another option is present:

• As part of the Homebrew formula for pipx, the formula could, either through use of userpath or through its own technique, ensure that the path is present as part of the pipx install.

To be sure, I had not anticipated this complication when devising this solution, but if you take a look at #131, you can see how the code substantially simplifies the logic... and in fact when the compatibility commits 13c8137c8f79b12a1d102d48832ae837988421c0 and a0790bbb8601a378b94e4da50e1440112ae376ed are reverted, the code will be even simpler, less duplicative, and the user experience more compatible by relying on userpath.

Moreover, if pipx is unable to rely on a tool like userpath, that implies that other projects would have to do the same, owning the UI and implementation or at the very least wrapping something like userpath. The benefits of decoupling these behaviors from pipx would apply to other projects as well.

Do you think any of the three approaches above could be viable so that pipx doesn't have to own its own copy of the "add a path to the user's PATH" functionality?

[cs01]

Thanks I understand what homebrew is doing now.

I don’t think it’s that big of a deal for pipx to maintain a command to call userpath. Userpath is importable, so pipx wouldn’t have to vendor anything, it could just replace the ensurepath code with userpath.append(). There may be some documentation updates necessary, but nothing that will cause long term headaches.

[jaraco]

I don’t think it’s that big of a deal for pipx to maintain a command to call userpath

I was considering that too... and even planning to draft a change. My only hesitation is that there will be for some time three different commands (ensurepath, userpath subcommand, and userpath command). After some time 'ensurepath' will go away, but which should be the preferred/recommended technique for users? Should pipx userpath be the recommended technique for Homebrew users only or for most users, and if the latter, does that imply that any other program that wishes to incorporate userpath should do the same?

[cs01]

Pipx ensurepath would come back and be the canonical method for all platforms and packaging tools. Its implementation would be userpath, but the user would not know or care.

[jaraco]

Pipx ensurepath would come back

I see. That approach brings other challenges:

• userpath and ensurepath don't implement equivalent interfaces. Would ensurepath adapt to better conform to the patterns expected by userpath (which may bring compatibility concerns), or would the ensurepath implementation be responsible for translating from its historical interface to the one implemented by userpath?
• This approach requires pipx to own and maintain this interface. One of the main motivations in adopting userpath's canonical interface was to offload that maintenance.
• This approach suggests that any other package that needs to do path manipulation should do the same, keeping a private copy of a possibly-subtly-divergent interface, and maintaining that interface. In other words, it's an anti-pattern.
• This approach abstracts away from the user the more general value of the userpath tool. By instead directing the user to invoke userpath directly, you're helping guide the users toward an approach that would have value in other use-cases, making their lives easier in ways we may not have imagined.

I feel there's real value in the original approach I've proposed... and I feel the Homebrew case shouldn't be imposing considerations on the design of a package like pipx, and I'd prefer to see if Homebrew can't accommodate the design.

Nevertheless, I've made my case and I'll defer to you Chad to decide how to proceed.

[jaraco]

As I think about it more, perhaps translating 'ensurepath' to invoke 'userpath' isn't that horrible. In fact, that was my original plan before embarking on this more ambitious change, so perhaps that's for the best.

[AlJohri]

@jaraco @cs01 lets try once to create the userpath homebrew package that pipx can depend on since this seems to be the cleanest solution. if there are issues with the notability requirements we can move forward with the pipx ensurepath?

I dropped a comment here inquiring: https://github.com/Homebrew/homebrew-core/pull/39483#issuecomment-489444236

[cs01]

Keeping the userpath command hermetic to pipx seems cleaner to me, but I don’t have a strong preference. It also allows pipx to define the cli api.

[cs01]

I will not be available the rest of today. Thanks for all your input and thoughts on this. I will check back tomorrow.

[AlJohri]

Looks like we are a go for creating a userpath homebrew formula 🎉 . @cs01 Regardless of your decision on wrapping or simply depending on the userpath cli, I'll go forward with making that formula. We can compare the UX after that.

[AlJohri]

Running into some issues with userpath at the moment: https://github.com/ofek/userpath/issues/3
Will circle back here when those are resolved.

[AlJohri]

pipx no longer relies on userpath being in the PATH so I'm closing this issue, although some of my original issues with userpath still stand, namely: https://github.com/ofek/userpath/issues/3#issuecomment-512973913