pip install -U does not remember whether a package was installed with --user

[pfmoore]

If you do

pip install --user dist
pip install -U dist

then the upgrade will uninstall the original installation from the user site-packages and install the upgraded version in the system site-packages. It should not do this, but rather should remember where the package being upgraded was located, and reinstall to the same location.

[pradyunsg]

And print a message about the fact that it's doing this?

[piotr-dobrogost]

This would require finding out what scheme (here it's the _user_ scheme) installed package uses. Is this feasible?

I think pip install --user -U dist is a _workaround_.

[RonnyPfannschmidt]

i think its valid to break there (early) instead of silently fixing user misstakes

[piotr-dobrogost]

It's not so clear this is a _user mistake_. Why does pip uninstall package installed using _user scheme_ during pip install -U dist while there's no --user option given? This is very bad UX. The only sane behavior would be to operate on packages installed in _user scheme_ only if --user was passed.

[RonnyPfannschmidt]

by break early i mean pip should bail out if install and uninstall locations don't match

the user mistake is not remembering --user
the pip ux failure is still working in unclear conditions

[pradyunsg]

I agree with @RonnyPfannschmidt.

If the user has installed a package with --user space, while upgrading/unistalling the package pip should require a --user. Basically, pip should not fiddle with the user space unless --user is passed.

[piotr-dobrogost]

the user mistake is not remembering --user

And the pip mistake is to uninstall package installed in _user_ scheme although there was no --user option passed to it.

by break early i mean pip should bail out if install and uninstall locations don't match

I'd rather have pip operate on one specific installation scheme (_user_ scheme if --user was passed) not across schemes so that install and uninstall locations would always match.

Basically, pip should not fiddle with the user space unless --user is passed.

Exactly and this is what I described saying _The only sane behavior would be to operate on packages installed in user scheme only if --user was passed._ I'm not sure @RonnyPfannschmidt agrees with this, however.

[RonnyPfannschmidt]

ignoring one of the locations when working on a specific package means there will be "interesting" effects when working with requirement files

which is why i prefer error out early, the later the user becomes aware of the mismatches,
the worse he/she will get hit with strange surprises

[pradyunsg]

I don't think pip should ignore it.

If the user tries to uninstall a package installed in system or venv scheme, with --user, pip should error out, with message about why it did so.

[pradyunsg]

@RonnyPfannschmidt What exactly do you mean the behavior should be like? Could you please give an example of it?

I'm not sure if I am correctly understanding what you are saying... Do you agree with the behavior in my previous comment?

[RonnyPfannschmidt]

i agree, pip should error out as soon as it sees a missmatch
this is in particular relevant when a requirements file installation finds a situation where bits are in user-site and other bits are in the system site

[pradyunsg]

Ohk. :bulb: There seems to be consensus on that behavior them. I have an idea/proposal...

AFAIK, there are 3 possible schemes for packages:

• "system" - for system/global packages (--system)
• "user" - for packages installed in user space (--user)
• "local" - for virtualenv packages (--local)

pip enforces a "working" scheme on every run. Outside a virtualenv, the default working scheme would be "system" (It should really be "user", that's another issue #1668). Inside a virtualenv, the default working scheme should be "local". Passing --system or --user or --local overrides the working scheme.

Only packages in the same scheme as the working scheme can be modified. By modifying, I mean installing or uninstalling a package. Trying to modify a package in a different scheme is not allowed and pip would print a message and error out.

So, modifying a package in system scheme with a "user" working scheme is not allowed. Nor is modifing a package in user scheme with a "system" working scheme. Niether are the other permutations with "local".

I think this results in a pretty simple behaviour model.

Any comments?

(editted)

[piotr-dobrogost]

Yes, that's exactly how I think this should work.

[ncoghlan]

To go back to the original scenario (and assuming the current "system" default scheme) and check I'm understanding the proposal correctly:

$ pip install --user dist
# Successfully installs to user site-packages
$ pip install -U dist
# Error like "Upgrade of 'dist' failed: target scheme (--system) does not match existing installation scheme (--user)"

This would need some logic to understand that user installs shadow system ones (so "pip install --upgrade --user dist" is fine, even if the only current installation is at system level), but the general idea sounds good to me.