Pip: VCS invocation merges stdout and stderr, resulting in output parsing error

Created on 2 Jan 2020 · 7Comments · Source: pypa/pip

Environment

pip version: 19.3.1
Python version: 2.7
OS: fedora

Due to a bug in mercurial, if both the split and evolve extension are enabled, mercurial will output a warning (on stderr) whenever it runs. When pip calls hg root to get the path of an editable install, pip captures both stdout and stderr, and so includes the warning as part of the path to the root of the repository and fails.

This can be reproduced with the following .hgrc:

[extensions]
evolve =
split =

and installing the evolve extensions pip install hg-evolve.

auto-locked

Source

tomprince

Most helpful comment

I think this is similar to what I observed in svn in https://github.com/pypa/pip/issues/7968 .

I have also created a PR for this (https://github.com/pypa/pip/pull/7969) which removes the behavior of vcs commands of merging stderr and stdout, and makes VCS commands use their own call_subprocess. Could you try it out and see if this fixes the hg issue?

deveshks on 5 May 2020

👍2

All 7 comments

It's correct that pip captures both stdout and stderr:
https://github.com/pypa/pip/blob/08f61a9b73287dfd155bc72bee0191895cf2819d/src/pip/_internal/utils/subprocess.py#L186-L191

Could you check if running hg --quiet root would silence the warning?

sbidoul on 3 Jan 2020

It does not appear to silence the warning.

tomprince on 9 Jan 2020

I’ll probably work on this after #7593. Some refactoring would improve the code touched by that PR, and would fix this bug en passant.

uranusjr on 15 Jan 2020

👀1

I’ve edited the title to be more generic. This also better reflects the problem (which is general, not limited to hg).

uranusjr on 5 May 2020

Copying some extra info from #8191:

In the git case this is pretty easy to reproduce by doing an export GIT_TRACE=1
The fact that stderr is redirected to stdout appears to be a historical accident. In 0cb2991cf0ef20f906ad3af67bd08c195d88d826 stderr was redirected to stdout to add the ability to filter tracebacks from stderr, and when that traceback filtering functionality was removed in f9429b0a98a5e193507f8453739d936f13cff412 stderr wasn't split back apart from stdout.