Pipx: Support module only packages

[gaborbernat]

Some packages don't offer console script for various reasons. We should be able to install these, perhaps by generating simpler wrappers that forward the invocation to the module. This could also version qualify installed scripts to ensure the user can explicitly differentiate between multiple version:

Think of:

• tox3.4
• tox3.6
• tox3.7
• tox (links to first target - 37?)

Reference: https://github.com/pypa/pep517/issues/54

[cs01]

Could you add more detail and examples in this issue so the feature request is self-contained here?

[AlJohri]

@gaborbernat can you just create your own pep517 cli wrapper and submit to pypi? then it can be installed through pipx normally

[gaborbernat]

Pulling in @FFY00 comments from 517:

How would this feature be useful?
This would be useful for projects that don't have an entrypoint but have a runnable module.

Describe the solution you'd like
Add a -m/--module argument to pipx install to install as a module and a -m argument to pipx to run the module.

Example

$ pipx install -m build  # install build as module
$ pipx -m build  # run the build module

The pip/Python equivalent

$ pip install build
$ python -m build

Related: FFY00/python-build#101

[FFY00]

Could we get some feedback on my proposed interface from the maintainers?

[pganssle]

Forgive my ignorance of how pipx works, but I think it might be worth thinking through exactly how the resolution is going to go.

Is the idea that when you _directly_ pipx install a module that exposes an executable module, pipx would introspect into the package and determine that this is the case, then record that those hooks are available in a certain virtualenv, but when you pipx install something that incidentally installs something with an executable module, pipx never exposes those hooks?

That seems relatively safe, unless you have multiple projects exposing executable modules with the same name (which I guess is already a problem anyway).

[pganssle]

This could also version qualify installed scripts to ensure the user can explicitly differentiate between multiple version:

One thing here: I don't really understand the relevance here. Is the multiple version thing a separate feature request, or is that relevant to module invocations?

[itsayellow]

Example

$ pipx install -m build  # install build as module
$ pipx -m build  # run the build module

I'm unenthusiastic about this interface. I'm not sure also about adding this complexity to pipx.

If we did do something like this, it seems much more appropriate to add module-running functionality only to pipx run

[FFY00]

Okay, would this work? Or were you thinking of something different?

$ pipx install -m build
$ pipx run -m build

Basically we add a -m/--module argument to install and run to run/install the module instead.

[itsayellow]

No, with pipx run you never need to use pipx install. It would just be pipx run -m build.

[FFY00]

Ah, sorry. What if you want to install from VCS or a sdist?

[itsayellow]

Then you would need to use e.g.

pipx run --spec git+https://<url...> -m build

[uranusjr]

The main problem I have with this is in the implementation. pipx does not currently know how to create entry points (but simply links/copies them), which is required for this feature to work. And the ability to do that is quite incolved.

[gaborbernat]

It doesn't have to create entry points. Just forward whatever the user feeds in via -m to the python interpreter it installed to. Isn't it?

[uranusjr]

How does pipx tell whether to forward the call to -m or to an executable? That would significantly complicate the install command in another way.

[pganssle]

@uranusjr I think it's not that tough. At install, pipx introspects the wheel for the package it was asked to install and determines which modules it provides that are executable, then it stores that information in some mapping between module names and virtual environments.

You may even be able to skip the part about "check if the module is executable", if you assume that pipx doesn't support directly installing two packages that expose the same module.

[FFY00]

Then you would need to use e.g.

pipx run --spec git+https://<url...> -m build

Isn't that wastefull and time consuming? If I wanted to install a custom project from sdist I would have to run specify --spec dist.tar.gz every time, that is not great UX.

@uranusjr I think it's not that tough. At install, pipx introspects the wheel for the package it was asked to install and determines which modules it provides that are executable, then it stores that information in some mapping between module names and virtual environments.

You may even be able to skip the part about "check if the module is executable", if you assume that pipx doesn't support directly installing two packages that expose the same module.

I agree with this, and it's what I envisioned when proposed.

[itsayellow]

Isn't that wastefull and time consuming? If I wanted to install a custom project from sdist I would have to run specify --spec dist.tar.gz every time, that is not great UX.

Yes it is a bit cumbersome, which is why we suggest you make an entry point so that we can easily install it. run is useful for scripts and/or simple pypi names, which is what you were originally suggesting.

One problem I have with the pipx -m <app> thing is that it just isn't how pipx is set up to work and isn't consistent with the way the pipx UX is setup. pipx isn't python, it's not an interpreter. To me this just seems awkward.

Also, it seems way easier to me to just to actually pick a name an entry point, instead of making a change to pipx so that it "introspects into packages to find which modules are runnable". Which of course is _possible_, but it adds a whole lot to maintain in pipx just to avoid picking the name of an entry point.