Tox: pipe output of command into file

[pytoxbot]

• Bitbucket: https://bitbucket.org/hpk42/tox/issue/73
• Originally reported by: mhubig
• Originally created at: 2012-12-03T12:31:33.883

Hi together,

I need to pipe the output of pylint into a file, so the Jenkins violations plugin can check it later, but unfortunately:

#!bash

commands = 
    pylint --rcfile=.pylintrc implib2 |tee pylint.out

gives me this error:

#!bash

|tee:1: [F] No module named |tee

and

#!bash

commands = 
    pylint --rcfile=.pylintrc implib2 > pylint.out

gives me this error:

#!bash

>:1: [F] No module named >

For now I put the pylint command into the (bash) build command of jenkins, but it would be nice if I could put this into my tox.ini file.

Cheers, Markus

[pytoxbot]

Original comment by @techtonik

Made PR for that - https://bitbucket.org/hpk42/tox/pull-requests/206/support-piping-command-output-to-file/diff and it worked for me on windows.

[pytoxbot]

Original comment by johnthagen

For reference, opened a ticket with Pylint to see if they will support output to a file natively: https://github.com/PyCQA/pylint/issues/1070

[pytoxbot]

Original comment by johnthagen

And for PyLint the --files-output option is deprecated and will be removed in Pylint 2.0, so that is no longer a long term option.

[pytoxbot]

Original comment by @witten

Here's a work-around for this issue:

commands =
    bash -c \'pylint --rcfile=.pylintrc implib2 |tee pylint.out\'

It's not pretty, but it works.

[pytoxbot]

Original comment by @hpk42

If you feel that adding support for redirection to files or pipes is easy, please submit a PR.

[pytoxbot]

Original comment by robvdl

flake8 have since added the --output-file option which has saved me in the end, but would it really be that hard to add support for a small amount of pipe characters (> and |, maybe >>)... Windows supports these too.

[pytoxbot]

Original comment by robvdl

I need this feature too, I can't get flake8 output piped to a file for jenkins

[pytoxbot]

Original comment by @techtonik

I am second to request this feature.

[pytoxbot]

Original comment by mhubig

Yes I can, but unfortunately it seems the only way to instruct pylint to write something into a file, is via the "files-output" option, but this results in a couple of files. Luckily the jenkins violations plugin can also handle this situation, by specifying a search wildcard like this: **/pylint_**.txt

But nevertheless I think it would be a nice feature to generally be able to collect the output of a command into a file.

[pytoxbot]

Original comment by @hpk42

tox doesn't implement shell semantics (it wouldn't easily work on windows). Can you instruct pylint with an option to write the output somewhere?

[obestwalter]

This has not been followed up for a long time and feels to me as trying to fix shortcomings of other tools with something that would turn tox into a maintenance nightmare. Closing for now.

[techtonik]

Probably there is something wrong with tox execution model if it manages other tools and requires modifying them to fit its own users scenarios.

If anybody is interested, here is a conflicting patch that I release in public domain - so feel free to pick it, adapt and submit as your own code. )
https://bitbucket.org/hpk42/tox/pull-requests/206/support-piping-command-output-to-file/commits

[obestwalter]

Probably there is something wrong with tox execution model if it manages other tools and requires modifying them to fit its own users scenarios.

I would rather say that every tool has its limitations and that limitations are not necessarily bad. Tox does not claim to have full shell functionality in the commands section. i don't know if it is feasible to try to add it and I don't know what it would break, but if you have a working alternative implementation of tox, I would be curious to try it. Your link goes only to a handful of unmerged patches. Do you have a fork somewhere, where these patches are integrated into a current version of the tox code?

[RonnyPfannschmidt]

as far as im concerned, its a good thing that tox doesn't claim/aim to be a full shell, in fact - you can use make or shell if tox doesnt fit your need (just use the externals)

implementing a full shell for tox is very likely a massive and senseless waste of volunteer time, because every step in that direction will bring up new breakages and requests - i think its fine to proclaim "NO"

[techtonik]

Note that the issue title is pipe output of command into file and not full shell implementation in tox.

[techtonik]

Do you have a fork somewhere, where these patches are integrated into a current version of the tox code?

Sorry. I am not using tox at all nowadays to port patches over to GitHub. My preferred workflow right now is LXD + setup.py test, but as you see above, setuptools want to get rid of this stuff in favor of tox, and inability to get full debug output for failed test run is a no-go for using tox for me.

[obestwalter]

inability to get full debug output for failed test run is a no-go for using tox for me.

I really fail to see the problem. What keeps you from piping the output of tox to a file? There must be something special about your workflow that I don't grasp.

[techtonik]

Because I want to see test results on the screen, and look to log only when they fail, and Windows doesn't have tee command. And that stuff is true for every Windows user.

[obestwalter]

Windows doesn't have tee command

Yes it does: https://technet.microsoft.com/en-us/library/dd347705.aspx

Since windows 10, you can even have a complete Linux subsystem - including bash and apt and whatnot: https://msdn.microsoft.com/de-de/commandline/wsl/install_guide

[techtonik]

@obestwalter okay, it is not true for every Windows user, but there is no powershell on my Windows system. Also, I don't want to learn PowerShell to use tox on Windows.

[obestwalter]

okay, I give up then.

[techtonik]

I can use tee on Windows, from git or whatever, but this issue is about embedding tee command into tox file, so that you can get logs from commands automatically without sophisticated howto.

The ticket is closed anyways.