Pip: Return Code Should be Non-Zero on Error

[ax3l]

Environment

• pip version: 20.0.2
• Python version: 3.6.9
• OS: Ubuntu 18.04 LTS

Description

Installs that error out with an unsatisfied requirement should return a non-zero exit status.

Expected behavior

Return non-zero exit code.

How to Reproduce

python3 -m pip install -U "sphinx<2" breathe

Output

# ...
ERROR: breathe 4.15.0 has requirement Sphinx>=3.0, but you'll have sphinx 1.8.5 which is incompatible.
Installing collected packages: sphinx, breathe
Successfully installed breathe-4.15.0 sphinx-1.8.5

$ echo $?
0

Related Issues

https://github.com/michaeljones/breathe/issues/495
https://github.com/michaeljones/breathe/issues/496

[sbidoul]

I believe this is blocked on #988. Currently, although pip detects the compatibility problem, it proceeds with installation, so in a way, it succeeds at installing something.

[vermeeren]

Even without a full dependency resolver it should probably still set the exit code to non-zero, or hard-fail right away when the problem is detected. This is more correct compared to completely ignoring this type of error.

Looking at sources exiting with non-zero or throwing something in _warn_about_conflicts() should be all that is needed to resolve this issue, see https://github.com/pypa/pip/blob/master/src/pip/_internal/commands/install.py#L510.

The user can then fix the issue by correcting pins in the dependencies and/or requirements.txt or similar. Some option such as --ignore-broken-deps could also be added to continue execution anyway, as is done now. But this should not be default.

so in a way, it succeeds at installing something.

Specifically about this: I do get what you mean, but this is not the expected behaviour for command-line utilities' exit code. For example if you call rm file_that_doesnt_exist file_that_exists it will complain about file_that_doenst_exist, still remove file_that_exists and in the end exit with a non-zero exit status since an error did occur during execution.

Considering this it probably makes sense to modify logger.critical() to set some error flag for the exit status, so that the exit status will always be non-zero if it is called at any point during execution of pip.

[sbidoul]

Changing the exit code in that situation would be a breaking change, as there are certainly people out there depending on pip not erroring in that case. This is the kind of change that the team working on the new resolver is going to manage as part of the transition plan.

[vermeeren]

Considering pip already prints ERROR, in all-caps even, this should be considered a bugfix, not an enhancement or similar. Running pip install some_package_that_doesnt_exist does result in exit status 1. I can get not throwing an exception and hard-failing, which makes it impossible to ignore errors.

Users that enjoy ignoring critical errors can either not set -e in their scripts, or ignore exit status explicitly by means such as pip ... || true depending on the used scripting language. Currently users' CICD fails very late with cryptic errors, or worse: wrong behaviour, because a package's dependencies were not satisfied, resulting in major incompatibilities between packages.

It is also difficult to spot that this is actually a dependency resolution problem, since the cryptic error produced at the failing step, which may even be inside a different CICD job, only include package's logs and/or stack traces. it does happen that these users will then open up issues on package's issue trackers, which causes some confusion for package maintainers.

As for users that depend on pip silently ignoring ciritcal errors and still returning exit status 0: is this really part of the interface guarantee within a major version? It reminds me of https://xkcd.com/1172/.

[rowanG077]

I'm not sure how #988 blocks this. Pip already detects if it fails it just doesn't set the return code properly.

[McSinyx]

@rowanG077, I believe what sbidoul meant was to answer to the specific case of unmet dependencies. IIUC,

1. pip is going to have proper dependency resolution
2. Until then, installation can have broken dependencies, and it is considered as a success.