tox doesn't write testing log

[techtonik]

One of the biggest tox drawbacks is that it doesn't write logs with errors that are presented to user. My scroll buffer is not enough to get to the first of failings tests. Example:

>tox
...
    if not path: raise CommandNotFound(program)
pbs.CommandNotFound: __path__

----------------------------------------------------------------------
Ran 2 tests in 0.107s

FAILED (errors=2)
ERROR: InvocationError: 'C:\\Documents and Settings\\user\\AppData\\Local\\Bin\...
___________________________________ summary ___________________________________
ERROR:   py27: commands failed
ERROR:   py36: commands failed

Grepping all files produced by tox, I can not find string pbs.CommandNotFound: __path__ anywhere.

[obestwalter]

@techtonik I was also wondering why this is not implemented, but then I realized that I never missed it either. Tox runs usually in a CI environment and there the output of tox becomes part of the log, so I guess nobody misses this?

Piping the output of the run tox into a file is always an option anyway, so I don't think this is in high demand.

[techtonik]

If tox is for CI, then what people should run for their tests during development?

[techtonik]

Second question - how to you run single test with tox? I remember there was a problem - not sure if it is fixed.

[obestwalter]

If tox is for CI, then what people should run for their tests during development?

People should do exactly what they want :)

I can only speak for myself. I use tox a lot locally and on CI and not only for testing, but also for building and releasing documentation and as a form of executable development workflow documentation. When I am developing I usually run pytest directly with a watch on that tests get rerun, if they are fast enough, but I also run tox before I push my changes (if I don't forget - otherwise my CI reminds me :)).

I never ran into the problem that I loose output of tox, as I have infinite scrollback in my terminals and if I really need to I just say tox 2>&1 tox.log.

I agree though that tox could have a switch to make it less talkative, there is also an issue about that: https://github.com/tox-dev/tox/issues/256

[techtonik]

I agree though that tox could have a switch to make it less talkative

I didn't raise that point. The point to run a single test is because tox is too slow and usually I leave it to run, and when it fails, there is no trace about what exactly failed, because scrollback buffer is full. Crippling ouput is not a solution, because there could be useful info. The solution is to write output to log in addition to screen reporting..

[obestwalter]

I didn't raise that point.

But I agree with that sentiment and that would also move towards a solution of the general "too much output" problem.

About your Windows specific problem: https://github.com/tox-dev/tox/issues/73#issuecomment-305782999

[nicoddemus]

The point to run a single test is because tox is too slow and usually I leave it to run,

There's no builtin support for that, but you can easily support it yourself by using {posargs}. For example pytest's tox.ini does this:

[testenv]
commands= pytest --lsof -rfsxX {posargs:testing}

Here {posargs} will be replaced by anything passed after -- in the commandline, or testing if nothing is given (as a default value). So it is possible to do this:

tox -e py27,py36 -- testing\acceptance_test.py -v

And tox will execute:

pytest --lsof -rfsxX testing\acceptance_test.py -v

In each test environment.

[techtonik]

you can easily support it yourself

Nice hack, except that it is not easy to discover.

[obestwalter]

Nice hack, except that it is not easy to discover.

What would constitute "easy to discover" for you? It's in the main part of the documentation after rall ... https://tox.readthedocs.io/en/latest/config.html#substitutions-for-positional-arguments-in-commands

[techtonik]

The whole name is unhuman. I would look for 'command line parameters to tox".

[obestwalter]

First hit, when googling for that phrase: http://tox.readthedocs.io/en/latest/example/general.html

[obestwalter]

So we're doing o.k. on the docs side for that bit, I guess 🎉

[techtonik]

I am tired. I don't want to use your ways of doing things. I just want the damn tool to write damn log into damn log directory that already exists for that purpose in each damn environment that it creates. Damn. )

[RonnyPfannschmidt]

so fork it and change it?

[gaborbernat]

maybe we can add a flag that would write the log also to the disk not just to the console?

(as far as running one test that is test runner dependent, so the {posargs} solution is the only general solution)

[techtonik]

Flag would help, but I would vote for a default behaviour, because tox already writes logs in .tox/log and .tox/py36/log etc. But there is nothing from tests being run.

[gaborbernat]

I would argue against default behavior, as that would change the behavior users got used too, and for one I would not use it at all, however I see the argument for it. Those files are internal logs, not for users.

[techtonik]

Those files are internal logs, not for users.

I am fine with it being internal, but default.

[gaborbernat]

We already do this for parallel builds, is that enough for you @techtonik?