Pip: Move all of pip's CLI handling into `pip._internal.cli`

[pradyunsg]

• Allows us to break up pip._internal.__init__
• Provides a clearer place to put all of the autocompletion support code.
• Better grouping of the code that does one thing

Suggested rearrangement (I'll omit pip._internal and say pip instead):

| before | after |
| --- | --- |
| pip.__init__.autocomplete() | pip.cli.autocompletion.autocomplete() |
| pip.basecommand | pip.cli.basecommand |
| pip.baseparser | pip.cli.option_parser |
| pip.cmdoptions | pip.cli.option_templates |
| pip.commands | pip.cli.commands |

[pradyunsg]

I'm fine if we skip renaming the modules tbh; that's mostly aesthetic. The moving of the modules should be enough of an improvement IMO.

[pradyunsg]

@pypa/pip-committers Does this look fine in principle?

[pfmoore]

Seems reasonable

[techalchemy]

Thanks for documenting this. We are keeping compatibility shims across everything since ~pip 8, I'm pretty confident that we have the common uses cases now. I can share that code if you like but I think we don't want to encourage that kind of usage?

I've implemented this set of shims + tooling in about 5 major projects now so I have a good sense of at least what our needs are in the packaging universe. Would it make sense to document what those are? I'm guessing a lot of that is going to be impossible to pull out of pip, but I also suspect other people are using those things...

As a side note the renames were intuitive except for option_templates

[pradyunsg]

With #5674 merged, I consider this done for the most part. Perhaps we can rename a file if the naming scheme seems odd (I went with a different ones from OP).

@techalchemy I am not sure what you are referring to here... Are you referring to the the removal of functionality/options? What do you mean by common use cases?

[techalchemy]

@pradyunsg things we are using from pips internals across multiple packages

[pradyunsg]

None the less, I think we're done here. I'll go ahead and close this issue.

---

@techalchemy I don't think that's something we'd want to encourage; using pip's internals externally. I personally would like to see the code though, for curiosity purposes. :)

[techalchemy]

@pradyunsg https://pip-shims.readthedocs.io/en/stable/quickstart.html#available-shims

[pfmoore]

@techalchemy Could you please at least add a huge disclaimer to the docs that makes it clear that the pip maintainers take no responsibility for any breakage as a result of using this package?

[techalchemy]

You bet @pfmoore — Worth noting is that I found copies of my much worse version of this code in numerous other applications that were already using various bits of pips internals so it seems people are still relying on this pretty heavily

[pfmoore]

... which frankly only makes me inclined to reorganise things again, just to make the point :smile:

I'm mostly joking, but seriously won't anyone look at what functionality these projects actually need, and work on putting together at least a design for some sort of API? There's no promise that anyone would actually implement it, but there's definitely no chance of that happening if no-one does. Your project is sort of a start at that, but as long as it's tied to the concepts that make up pip's internals, it's of limited help.

Yeah, I know, volunteer effort, limited time, expedient solutions :smile: And given that no-one has complained at pip over API stability in this area (it's all been about pip.main!) there's no reason for me to worry about it. Just needed to get the comment off my chest. (And I'm very definitely not having a go at you over this - quite the opposite as you're acting as a buffer for us).

[techalchemy]

Yeah the whole reason for making this project was sort of as a documentation step toward showing at least what we are using (and I guess people copy us for some reason). We just finished writing a resolver (well almost finished) so it was on my mind

I originally wrote this for piptools which provides the backing implementation currently in pipenv. I’m just not sure what is specific to our use case vs what can be extracted into a more general library. The broader question I guess is about the optimal division of responsibility and what the resulting libraries should be. I’m not too sure beyond the fact that there should be a resolver and an installer and something should query the index. Mostly whenever I’m reading pip internals im just glad you guys had to deal with most of the system level problems already

[techalchemy]

(Giant warning added btw)