Pip: Pip continues installing despite conflicting dependencies

[rhpvorderman]

Environment

• pip version: pip 20.0.2
• Python version: Python 3.7.3
• OS: Debian GNU/Linux 10

Description
if a package install leads to conflicting packages in the environment pip continues installing and happily exits with exit code 0: everything is fine!

If you then run of the package everything breaks because of the interface of dependencies not being there, or worse continues without error and output wrong results!

Expected behavior
Two options:

• Pip exits with a non-zero exit code. The package install does NOT happen.
• Pip prompts the user for input, install package A breaking B, or the other way round, or abort.

How to Reproduce

• Create a new environment virtualenv --python python3 conflicting-env
• activate it: source conflicting-env/bin/activate
• run pip install pytest-workflow==1.3.0 'pytest<5'
• run pytest and watch how everything breaks down.

Output

(conflicting-env) $ pip install pytest-workflow==1.3.0 'pytest<5'
Collecting pytest-workflow==1.3.0
  Downloading pytest_workflow-1.3.0-py3-none-any.whl (36 kB)
Collecting pytest<5
  Using cached pytest-4.6.9-py2.py3-none-any.whl (231 kB)
Processing ./.cache/pip/wheels/5e/03/1e/e1e954795d6f35dfc7b637fe2277bff021303bd9570ecea653/PyYAML-5.3.1-cp37-cp37m-linux_x86_64.whl
Collecting jsonschema
  Using cached jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting atomicwrites>=1.0
  Using cached atomicwrites-1.3.0-py2.py3-none-any.whl (5.9 kB)
Collecting py>=1.5.0
  Using cached py-1.8.1-py2.py3-none-any.whl (83 kB)
Collecting importlib-metadata>=0.12; python_version < "3.8"
  Using cached importlib_metadata-1.6.0-py2.py3-none-any.whl (30 kB)
Collecting more-itertools>=4.0.0; python_version > "2.7"
  Using cached more_itertools-8.2.0-py3-none-any.whl (43 kB)
Collecting packaging
  Using cached packaging-20.3-py2.py3-none-any.whl (37 kB)
Collecting wcwidth
  Using cached wcwidth-0.1.9-py2.py3-none-any.whl (19 kB)
Collecting pluggy<1.0,>=0.12
  Using cached pluggy-0.13.1-py2.py3-none-any.whl (18 kB)
Collecting six>=1.10.0
  Using cached six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting attrs>=17.4.0
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Processing ./.cache/pip/wheels/22/52/11/f0920f95c23ed7d2d0b05f2b7b2f4509e87a20cfe8ea43d987/pyrsistent-0.16.0-cp37-cp37m-linux_x86_64.whl
Requirement already satisfied: setuptools in ./conflicting-env/lib/python3.7/site-packages (from jsonschema->pytest-workflow==1.3.0) (46.1.3)
Collecting zipp>=0.5
  Using cached zipp-3.1.0-py3-none-any.whl (4.9 kB)
Collecting pyparsing>=2.0.2
  Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
ERROR: pytest-workflow 1.3.0 has requirement pytest>=5.4.0, but you'll have pytest 4.6.9 which is incompatible.
Installing collected packages: pyyaml, attrs, six, pyrsistent, zipp, importlib-metadata, jsonschema, atomicwrites, py, more-itertools, pyparsing, packaging, wcwidth, pluggy, pytest, pytest-workflow
Successfully installed atomicwrites-1.3.0 attrs-19.3.0 importlib-metadata-1.6.0 jsonschema-3.2.0 more-itertools-8.2.0 packaging-20.3 pluggy-0.13.1 py-1.8.1 pyparsing-2.4.7 pyrsistent-0.16.0 pytest-4.6.9 pytest-workflow-1.3.0 pyyaml-5.3.1 six-1.14.0 wcwidth-0.1.9 zipp-3.1.0

Note how pip does notice that it installs conflicting packages, but then continues anyway. In automated environments this can lead to lots of problems. Automated scripts will not notice something is amiss and will crash or much worse: deliver faulty data.

[McSinyx]

Hi there, you are here just in time to try out v20.1b1 which has a more proper dependency resolver that should be able the conflict gracefully (GH-8099). The legacy resolver, however, is trying its best, and I believe the reason for it is to make it possible to install/upgrade a new package even if that breaks other ones. I'm truly sorry for your inconvenience.

[rhpvorderman]

The legacy resolver, however, is trying its best, and I believe the reason for it is to make it possible to install/upgrade a new package even if that breaks other ones.

This is good fallback behavior. But I do not get why this is the default behavior. Surely a program should just exit with non-zero if things are wrong? Unless some --continue-on-error flag is given explicitly by the user.

Hi there, you are here just in time to try out v20.1b1 which has a more proper dependency resolver that should be able the conflict gracefully (GH-8099).

I will check it out, but I see that it won't be used by default for quite some time. So this won't solve the problem in a lot of CI and other environments. The workaround is appreciated though :+1: .

[McSinyx]

I have neither the experience nor the authority to properly explain the decision to you, but you can check out GH-7744 and GH-7846 for relevant discussions. I can recall an exactly the same issue with this opened just a few days ago, but unfortunately I couldn't find it :disappointed:

[rhpvorderman]

No problem. Thanks for the info!

[pfmoore]

But I do not get why this is the default behavior.

Mostly, it's historical and it's a bug. It's just that we're not going to fix it in the old resolver at this point, as the new one is nearly ready and we plan on not having this issue in the new resolver in the first place 🙂

[McSinyx]

Mostly, it's historical and it's a bug. It's just that we're not going to fix it in the old resolver at this point

Thanks for the explanation :blush: I think I've been spreading inaccurate information for quite a while.

[rhpvorderman]

@pfmoore Thanks for the context. I am curious how the new resolver will work out.

[pfmoore]

I think I've been spreading inaccurate information for quite a while.

I doubt it. The whole question is very complex. The current behaviour of spotting that we've hit a conflict and having no means of backing out and trying alternatives is part of why the old resolver needs to be replaced. In itself, that's just "yeah, it's broken but to fix it needs a rewrite" that is what started the "pip needs a new resolver" process. The part about reporting the conflict and continuing is more or less "by design" because there's not really a better option (crashing out leaves people with no good way to address the issue, and in too many cases there is a good set of packages to install, it's just that pip can't find them). It's bad, but the available alternatives in the old resolver don't seem much better. The bit where we report "ERROR" but then return an exit status of 0 is hard to justify, so I'd class that as a bug. But I reserve the right to fix it by changing the text to "WARNING" 🙂

In the new resolver, we will fix this, but by various means:

1. The new resolver should find a workable solution in many more cases.
2. We have design decisions that we haven't finalised yet on how we handle edge cases (see #7744 for an example).
3. We'll (hopefully!) fail properly when we can't find a solution.

As yet, we have no real view on how much control we'll give to the user to override in the case of conflicts. That's a whole other area that needs more experience and user feedback (we have research going on to help inform that process).

So... not so much inaccurate information as "way more complicated than can be summarised in a quick comment" 🙂

[Yajo]

In the mean time, as a workaround:

pip install -r conflicting-requirements.txt
pip check

pip check will fail if there are broken dependencies.

[pradyunsg]

Thanks for filing this issue @rhpvorderman!

You're hitting #988 -- an issue that's on pip's donor funded roadmap for this year. I see that both pip check as well as the relevance of the work on the new resolver have been mentioned here already, so I don't think there's much to add on top.

Basically, this specific issue will be fixed as part of #988. I'm going to go ahead and close this as a duplicate of that issue now, since there's no reason to tracking the same problem in multiple issues. :)

[rhpvorderman]

Thanks @pradyunsg !