Tox: handle broken pipe for commands

[gaborbernat]

tox --showconfig | head -n 0

Traceback (most recent call last):
  File "../../github/tox/.tox/py27/bin/tox", line 11, in <module>
    load_entry_point('tox', 'console_scripts', 'tox')()
  File "/Users/bgabor8/git/github/tox/src/tox/session/__init__.py", line 43, in cmdline
    main(args)
  File "/Users/bgabor8/git/github/tox/src/tox/session/__init__.py", line 68, in main
    exit_code = session.runcommand()
  File "/Users/bgabor8/git/github/tox/src/tox/session/__init__.py", line 185, in runcommand
    self.showconfig()
  File "/Users/bgabor8/git/github/tox/src/tox/session/__init__.py", line 268, in showconfig
    show_config(self.config)
  File "/Users/bgabor8/git/github/tox/src/tox/session/commands/show_config.py", line 21, in show_config
    report.line("  {:<15} = {}".format(attr.name, getattr(envconfig, attr.name)))
  File "/Users/bgabor8/git/github/tox/src/tox/reporter.py", line 93, in line
    self.logline("line", msg, **opts)
  File "/Users/bgabor8/git/github/tox/src/tox/reporter.py", line 83, in logline
    self.tw.write(line_msg, **opts)
  File "/Users/bgabor8/git/github/tox/.tox/py27/lib/python2.7/site-packages/py/_io/terminalwriter.py", line 256, in write
    write_out(self._file, markupmsg)
  File "/Users/bgabor8/git/github/tox/.tox/py27/lib/python2.7/site-packages/py/_io/terminalwriter.py", line 421, in write_out
    fil.flush()
IOError: [Errno 32] Broken pipe

[gnikonorov]

Hey @gaborbernat, could I fix this one too?

[gaborbernat]

Yes 👍

[gnikonorov]

Looking at the stack trace a bit more this is coming from the py library, and has nothing to do with the source of this repo since all we do is call terminal writer's write method.

The way I see it there are 2 options here:

• Submit a patch to py.
• Migrate away from py's terminal writer to a custom tox one. Pytest did this. We could probably just copy over the files from pytest, and make any patches we need ( if any are even needed. I haven't checked ). The benefit I see here is that we remove the dependency on a maintenance mode library, which this project will probably have to do at some point in the future anyway.

How would you like to proceed @gaborbernat ?

[gnikonorov]

FWIW I think it would make sense to do option 2. It's a non breaking change, and I think that it wouldn't be too involved.

[gaborbernat]

Oh if it's that complicated probably won't happen. py is no longer maintained. tox 4 is in development on the rewrite branch and that no longer uses it, so will be solved automatically by it. https://github.com/tox-dev/tox/tree/rewrite#rewrite-branch

[gnikonorov]

Thanks for clarifying @gaborbernat. Unfortunately ( or fortunately ) there is no easier solution. This isn't a tox bug

[gaborbernat]

Thanks for the investigation. Hopefully, tox 4 will get to an rc by Christmas and fixes this. I can confirm that on the rewrite branch we no longer have this issue:

tox4 c -e py39 | grep commands
commands =
commands_pre =
commands_post =

[gnikonorov]

Thanks for the investigation. Hopefully, tox 4 will get to an rc by Christmas and fixes this. I can confirm that on the rewrite branch we no longer have this issue:

Of course @gaborbernat, looking forward to tox 4! Is there any way I can contribute?

Also, it's worth noting that piping tox to grep won't trigger this ( I'm not exactly sure why ). I don't think that matters for v4 since the dependency on terminalwriter is gone, but for the sake of completeness here is the current behavior of piping to grep v piping to head:

gnikonorov:~/tox$ tox c -e py37 | grep commands
py37 run-test: commands[0] | pytest --cov /home/gnikonorov/tox/.tox/py37/lib/python3.7/site-packages/tox --cov-config /home/gnikonorov/tox/tox.ini --junitxml /home/gnikonorov/tox/.tox/junit.py37.xml -n=auto c
ERROR:   py37: commands failed
gnikonorov:~/tox$ tox c -e py37 | head -n 0
Traceback (most recent call last):
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/session/__init__.py", line 225, in subcommand_test
    run_sequential(self.config, self.venv_dict)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/session/commands/run/sequential.py", line 18, in run_sequential
    installpkg(venv, venv.package)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/session/commands/run/sequential.py", line 46, in installpkg
    venv.installpkg(path, action)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/venv.py", line 396, in installpkg
    self.install_pkg(sdistpath, action, "inst")
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/venv.py", line 382, in install_pkg
    action.setactivity("{}-nodeps".format(name), dir)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/action.py", line 66, in setactivity
    reporter.verbosity0("{} {}: {}".format(self.name, name, msg), bold=True)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/reporter.py", line 115, in verbosity0
    self.logline_if(Verbosity.DEFAULT, "verbosity0", msg, **opts)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/reporter.py", line 76, in logline_if
    self.logline(of, message, **kwargs)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/reporter.py", line 84, in logline
    self.tw.write(line_msg, **opts)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/py/_io/terminalwriter.py", line 256, in write
    write_out(self._file, markupmsg)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/py/_io/terminalwriter.py", line 421, in write_out
    fil.flush()
BrokenPipeError: [Errno 32] Broken pipe

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/gnikonorov/.local/bin/tox", line 11, in <module>
    sys.exit(cmdline())
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/session/__init__.py", line 44, in cmdline
    main(args)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/session/__init__.py", line 69, in main
    exit_code = session.runcommand()
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/session/__init__.py", line 197, in runcommand
    return self.subcommand_test()
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/session/__init__.py", line 227, in subcommand_test
    retcode = self._summary()
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/session/__init__.py", line 251, in _summary
    reporter.separator("_", "summary", reporter.Verbosity.QUIET)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/tox/reporter.py", line 71, in separator
    self.tw.sep(of, msg)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/py/_io/terminalwriter.py", line 243, in sep
    self.line(line, **kw)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/py/_io/terminalwriter.py", line 271, in line
    self.write(s, **kw)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/py/_io/terminalwriter.py", line 256, in write
    write_out(self._file, markupmsg)
  File "/home/gnikonorov/.local/lib/python3.6/site-packages/py/_io/terminalwriter.py", line 421, in write_out
    fil.flush()
BrokenPipeError: [Errno 32] Broken pipe
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='UTF-8'>
BrokenPipeError: [Errno 32] Broken pipe
gnikonorov:~/tox$

[gaborbernat]

Is there any way I can contribute?

Absolutely. We're still in the phase of trying to make it CLI interface/behavior compatible with tox 3, so any improvements on this front are much appreciated. You can open PRs against the rewrite branch. However, if you pick up something let me know before, so we don't work in parallel on it. As starting my recommendation would be to add tests, we're at 82% and way away from 100%, and this would allow you to get to know the code, how it's broken up and such.

[gnikonorov]

Absolutely.

Thank you @gaborbernat, looking forward to contributing! I'll start with some tests then this weekend

[gaborbernat]

I'll close this now as it's fixed in tox 4.