Pip: Provide a way to supress progressbar, but display exceptions

[guettli]

I we provide the --quiet option, pip does not display exceptions like this:

        install_requires=get_install_requires(),
      File "/tmp/pip-build-QuFDf8/selectable-tbz/setup.py", line 7, in get_install_requires
        return [str(entry.req) for entry in requirements if not entry.editable]
      File "/home/modwork_vums_di995/lib/python2.7/site-packages/pip/req/req_file.py", line 31, in parse_requirements
        "parse_requirements() missing 1 required keyword argument: "
    TypeError: parse_requirements() missing 1 required keyword argument: 'session'

Please give us an option to supress the progressbar, but display exceptions.

Thank you.

[dstufft]

Hmm, the quiet option shouldn't hide errors.

I'm ok with ask option to hide the progress bar though.

On May 6, 2015, at 4:57 AM, Thomas Güttler [email protected] wrote:

I we provide the --quiet option, pip does not display exceptions like this:

    install_requires=get_install_requires(),
  File "/tmp/pip-build-QuFDf8/selectable-tbz/setup.py", line 7, in get_install_requires
    return [str(entry.req) for entry in requirements if not entry.editable]
  File "/home/modwork_vums_di995/lib/python2.7/site-packages/pip/req/req_file.py", line 31, in parse_requirements
    "parse_requirements() missing 1 required keyword argument: "
TypeError: parse_requirements() missing 1 required keyword argument: 'session'

Please give us an option to supress the progressbar, but display exceptions.

Thank you.

—
Reply to this email directly or view it on GitHub.

[guettli]

I just want my use case to be solved. If --quiet would show the traceback, it would be a good solution for me.

[squfrans]

i can confirm that -q hides errors. this is trying to install cffi without libffi-dev present:

$ pip -q install cffi
/var/venv/devserver/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:79: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
    Command "/var/venv/devserver/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-IENjBH/cffi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-5k82OS-record/install-record.txt --single-version-externally-managed --compile --install-headers /var/venv/devserver/include/site/python2.7/cffi" failed with error code 1 in /tmp/pip-build-IENjBH/cffi

this is somewhat annoying, because without -q it is impossible to see why it failed,
as the build logs in /tmp are also removed. but without -q the progress bar is always there
creating a huge amount of noise in tools like jenkins and vagrant. these tools always
emulate a full tty, so relying on stdout being a non-tty does not cut it anymore,
making the reasoning in #2369 moot.

[virtuald]

+1 on providing a mechanism to disable the progressbar even with a TTY

[yoavram]

+1

[suvit]

+1

[ajlanghorn]

Whilst this isn't painful for me right now, having the progress bars output is a bit of a pain since I'd quite like Pip to be used quietly in the background and only error out on exceptions since I catch exit codes. So, this is a +1 from me!

[edmorley]

It would still useful to be able to disable the progress bar without having to resort to --quiet - for example on Travis where the output is streamed (so appears to pip as a TTY), it currently results in logs that are filled with progress bar cruft (eg https://s3.amazonaws.com/archive.travis-ci.org/jobs/130634491/log.txt - and see also travis-ci/travis-ci#1337).

I'm happy to open a PR for this -- I was thinking there could be both a command line switch and also an equivalent environment variable (that people like Travis CI could set by default).

Any preference as to naming? eg:
--disable-progress
--no-progress

And for the environment variable:
PIP_DISABLE_PROGRESS
PIP_DISABLE_PROGRESS_BAR

Thanks! :-)

[dstufft]

A PR would be fine with me, though you don't need to do anything special for an environment variable, pip options automatically become environment variables.

I would maybe suggest something like --progress-style=off / ascii / pretty or something like that, but I'm not sure if it's worth it to allow people to downcast the fancier progress bar into an ASCII progress bar (it will automatically use ASCII only if it doesn't think the pretty one will be displayed correctly).

[memory]

+1 on this. Hilariously, I ran into this issue in the _exact_ same context: installation of cffi failed in a jenkins build that didn't have libffi.so available, and --quiet suppressed the error completely.

Adding an option to disable only the progress bar would be nice, but having --quiet suppress stack traces / fatal error output from inside pip seems like an outright bug. If I wanted to suppress all output including fatal errors, I already know how to use 2>&1 >/dev/null :)

[alexch]

having --quiet suppress stack traces / fatal error output from inside pip seems like an outright bug.

yeah, seems this should be two issues (both of which I'd "me too")

[witte-de-with]

+1, see also @AvnerCohen 's comment here https://github.com/pypa/pip/issues/2369#issuecomment-224853759

[bialix]

+1 - installing somethig with pip on remote host via fabric is really a lot of mess because fabric shows every new state of progress bar on new line, below just a tail of the output. This is also related to https://github.com/pypa/pip/issues/2449 (re disabling ansi colors):

out: ←[K    94% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦП | 3.3MB 50.1MB/s eta 0:00:01
out: ←[K    94% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦО | 3.3MB 50.1MB/s eta 0:00:01
out: ←[K    94% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦН | 3.3MB 48.2MB/s eta 0:00:01
out: ←[K    95% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦМ | 3.3MB 48.4MB/s eta 0:00:01
out: ←[K    95% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦМ | 3.3MB 48.6MB/s eta 0:00:01
out: ←[K    95% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦЛ | 3.3MB 47.8MB/s eta 0:00:01
out: ←[K    96% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦК | 3.3MB 48.3MB/s eta 0:00:01
out: ←[K    96% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦЙ | 3.4MB 48.1MB/s eta 0:00:01
out: ←[K    96% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИ | 3.4MB 48.5MB/s eta 0:00:01
out: ←[K    96% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИ | 3.4MB 48.7MB/s eta 0:00:01
out: ←[K    97% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИ | 3.4MB 47.9MB/s eta 0:00:01
out: ←[K    97% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦП| 3.4MB 48.4MB/s eta 0:00:01
out: ←[K    97% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦО| 3.4MB 49.3MB/s eta 0:00:01
out: ←[K    98% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦН| 3.4MB 49.3MB/s eta 0:00:01
out: ←[K    98% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦМ| 3.4MB 49.9MB/s eta 0:00:01
out: ←[K    98% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦЛ| 3.4MB 50.1MB/s eta 0:00:01
out: ←[K    98% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦЛ| 3.5MB 50.2MB/s eta 0:00:01
out: ←[K    99% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦК| 3.5MB 50.3MB/s eta 0:00:01
out: ←[K    99% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦЙ| 3.5MB 49.4MB/s eta 0:00:01
out: ←[K    99% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИ| 3.5MB 49.9MB/s eta 0:00:01
out: ←[K    100% |тЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИтЦИ| 3.5MB 411kB/s
out: ←[?25hInstalling collected packages: virtualenv
out: Successfully installed virtualenv-15.0.3
out:

And no - I can't disable tty due to sudo settings restrictions.

[memory]

@bialix FWIW the workaround I discovered for scenarios like fabric and jenkins execution of pip is to connect stdout to a pipe -- even if that pipe just in turn connects to its own stdout, it's enough to suppress the progress meters:

pip install -U virtualenv | cat

Dumb, but it works. See also pty=False when using Fabric.

[dmytrokyrychuk]

@memory the workaround does not preserve the pip command exit code.

[piotr-dobrogost]

@orgkhnargh
Get exit status of process that's piped to another

[dmytrokyrychuk]

@piotr-dobrogost Didn't know that, thank you! But I think that stacking more and more workarounds will add unnecessary noise to CI scripts. Personally I'd prefer pip install --no-progress package over pip install package | cat && exit ${PIPESTATUS[0]}.

[pfmoore]

Agreed an option for this would be good. As has been noted, a PR for this would be good, although no-one has taken up that suggestion yet. If anyone does raise a PR, ping me and I'll be happy to review it.

[xavfernandez]

This should be fixed by #4194

[ssbarnea]

Why was this bug closed? The linked merged PR does not really add --no-progress-bar option.

[pradyunsg]

@ssbarnea --progress-bar off

[ssbarnea]

@pradyunsg Sorry but I get no such option: --progress-bar even with latest release (9.0.3) of pip.
This bug was closed more than an year ago, so I supposed that it should have being included in a release.

I also did a pip install --help | grep progress and got nothing, so I suspect that this feature was never released.

[AvnerCohen]

I think this is part of the current main dev version, 10.0.0 (https://github.com/pypa/pip/tree/release/10.0.0), 9.0.x is at maintenance mode with no new features.

https://mail.python.org/pipermail/distutils-sig/2017-October/031642.html

[ssbarnea]

@AvnerCohen Thanks for the update. In this case it would be extremely useful to populate the Milestone field with 10.0.0 value, so others would not confuse

@pradyunsg Can you please help with Milestone field? Contributors cannot edit this fields, only Members can.

[memory]

A little more forcefully this time since it clearly got lost in the mess above (two years ago, sigh):

Regardless of whether pip9 allows turning the progress bar off or not, the fact that --quiet suppresses FATAL ERRORS AND STACK TRACES cannot reasonably be described as anything other than a bug.

The linked message from the distutils-sig mailing list suggests that the full release of pip 10 is "one or two months away" in...October of 2017 (nearly five months ago). And even once 10 comes out of beta, it can be safely assumed that given the API differences that many shops will continue using pip9 for months or years to come as they prioritize their conversions.

@AvnerCohen if I managed to put together a PR for pip9 that fixed only this misbehavior, is there any chance at all that it would be accepted and released?

[pfmoore]

@memory As @dstufft said above:

Hmm, the quiet option shouldn't hide errors.

This issue is pretty confused, probably because the title refers to suppressing the progress bar (which is covered in pip 10 via --progress-bar=off.

If you have a reproducible example of --quiet suppressing errors, then please raise a new issue for it, so that it doesn't get lost in the noise again. Unfortunately, we are now in the final stages of releasing pip 10 (expected in 2 weeks, barring major issues in the beta) so this won't get picked up until after that, but we will look at it then. We aren't doing bugfix releases for the 9.x series, so any fix for this will only be available for pip 10.

[memory]

The title of this bug refers to the progress bar, but I'll note that the original bug _text_ refers to --quiet suppressing fatal errors back in May of 2015 and is confirmed multiple times in the comments below.

Anyway I'll stop grousing and install 10.0.0b2, see if it's reproducible there and file a ticket if so.

[est]

@edmorley
is the environ PIP_DISABLE_PROGRESS_BAR still available?

Because I am trying to write a script like

pip install --progress-bar off --upgrade pip

Then pip says no such option: --progress-bar because of old version 9

I think using a envion switch is better for compatibility

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.