Pip: Progress bar bug breaks torch installation on Travis CI

[adamjstewart]

Environment

• pip version: 18.1
• Python version: 3.5.6, 3.6.7, 3.7.1
• OS: Ubuntu Xenial
• CI: Travis

Description

Installing torch with pip on Travis CI results in the following error:

Collecting torch>=0.4 (from -r requirements.txt (line 7))
Downloading https://files.pythonhosted.org/packages/f5/3b/0b8de6e654c2983898564226792c6f09d9bcaba97b7b29c40e4ed4ae43ed/torch-1.0.0-cp37-cp37m-manylinux1_x86_64.whl (591.8MB)
73% |███████████████████████▌ | 434.3MB 36.8MB/s eta 0:00:05
The job exceeded the maximum log length, and has been terminated.

Expected behavior

This shouldn't be problematic. All other Python dependencies were successfully installed via pip on Travis CI.

How to Reproduce

1. Create an empty GitHub repository
2. Add a .travis.yml file that tries to pip install torch during the install phase
3. Add Travis CI via GitHub Apps integration and initiate a build

Output

One of the times I submitted, I got lucky and saw the following log:

It appears that there is a bug in the progress bar. This may only affect very large downloads like torch.

Known Workarounds

The following workaround solved the problem for me:

$ pip install --progress-bar off ...

It would be good to get the progress bar fixed though.

[native-api]

It looks like Travis doesn't handle terminal control characters the same way as a real terminal would: after seeing BS and CR, it doesn't move a virtual "caret" backwards and overwrite characters on the current line with the following output.

So, this seems like Travis' problem because without secret variables, Travis reports itself as an xterm terminal, thus giving pip the right to use terminal control characters.

[adamjstewart]

@native-api pip's progress bar works just fine for other packages. It only failed for torch. Does the progress bar behave any differently for large downloads?

[native-api]

The progress bar output in the Travis log looks just as "broken" for other downloads, too. The only difference is they don't take as long, so the amount of garbage in the log is insufficient to go over the log size limit.

[adamjstewart]

Well that explains it! How can we go about resolving this? Does something need to change on the Travis CI side of things? I already opened an issue here: https://travis-ci.community/t/large-pip-installs-exceed-maximum-log-length/1599/3. Feel free to comment, or I can relay your comments to them.

[native-api]

Your workaround is okay in my book (but the documented form is --progress-bar off).

This is just how a progress bar is implemented in console programs -- it continuously overwrites itself using terminal control characters.

Possible solutions from Travis side would be to add ability to somehow instruct Travis to (temporarily) handle BS and CR like a terminal would -- i.e. overwrite the log (e.g. travis_terminal <command>). Or to report itself to programs differently (e.g. report no carriage-return and backspace). The latter is very problematic (most programs now silently assume and expect xterm capabilities at least), the former is less so (just opens a possibility to lose some output).

[adamjstewart]

Thanks, I'll relay this to them.

The workaround works fine, but I'm sure other Travis users would expect the default pip install to Just Work™. Hopefully they can do something on their end.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.