When piping pytests output e.g. into tee it will get buffered.
To work around this, you can e.g. export PYTHONUNBUFFERED=1 or add -u to the shebang line of the pytest script (see http://stackoverflow.com/questions/107705/disable-output-buffering).
I think it makes sense to enable it by default.
blueyed
All 20 comments
I disagree - output buffering is there for a reason, and basically every application buffers its output by default. I don't think there's a good reason to do anything different than that.
The-Compiler
on 29 Aug 2016
IIRC the buffering does not happen when running on a terminal anyway (the common use case).
It is unusual to redirect its output, and when this gets done nobody relies on or benefits really from it being buffered?!
blueyed
on 29 Aug 2016
Isn't the "normal" unix behaviour to detect if it's being run in a pipeline and then disable the buffering? In fact I'm surprised python does not behave like that by default. Unless there's something we do in pytest which actually disables this? If so I'd say this is still a bug, we should behave fine either on terminal or in a pipe.
flub
on 5 Sep 2016
this is a python bug
RonnyPfannschmidt
on 5 Sep 2016
So we could disable the output buffering explicitly then?
FWIW, to make control characters (e.g. cursor keys, but also C-a) work with pdb.set_trace(), you need to use script around pytest/Python anyway, which then also disables the output buffering:
script -q -c 'pytest …' /dev/null | tee /tmp/foo
this is handled at the python level, - we can enforce flushes
what use case are you trying to implement?
RonnyPfannschmidt
on 5 Sep 2016
My use case is using https://github.com/janko-m/vim-test with https://github.com/radenling/vim-dispatch-neovim: vim-dispatch-neovim uses a Neovim terminal instance and that causes lines to be wrapped at its width (which might be really narrow in a tmux split etc).
Therefore I've thrown in | tee there to capture pytest's output into a file directly (instead of reading it from the terminal buffer) - https://github.com/radenling/vim-dispatch-neovim/issues/7.
blueyed
on 5 Sep 2016
just do a python -B -m pytest ?
RonnyPfannschmidt
on 6 Sep 2016
i mean for that use case py.test should have a non-gui stdout output format that allows to present things nicely
ideally flushed json-lines or anything that can be piped to a presentation layer directly without enforcing the normal output text limitations
RonnyPfannschmidt
on 6 Sep 2016
@RonnyPfannschmidt
How is -B relevant her?
-B Don't write .py[co] files on import. See also PYTHONDONTWRITEBYTE‐ CODE.py.test should have a non-gui stdout output format that allows to present things nicely
Agreed.
blueyed
on 6 Sep 2016
whops, my error, i meant -u
RonnyPfannschmidt
on 6 Sep 2016
@blueyed can this be closed as upstream python bug?
RonnyPfannschmidt
on 18 Jan 2017
@RonnyPfannschmidt
Not sure?!
After all we could set pytest up accordingly, and it could be argued to not change this in Python by default.
python -B -m pytest might be a good enough workaround though..
blueyed
on 18 Jan 2017
@blueyed still an issue for pip/a pep
RonnyPfannschmidt
on 19 Jan 2017
@RonnyPfannschmidt
What do you mean? That it's not a pytest issue?
pytest might be able to set this up in its (entrypoint) script..
blueyed
on 19 Jan 2017
@blueyed no, setuptools/pip doesn't support that in entry-point scripts and i'm inclined to NOT invent it because it will be an maintenance pain wit regular platform issues
RonnyPfannschmidt
on 20 Jan 2017
Please also note that python supports setting this through an PYTHONUNBUFFERED environment variable:
-u : unbuffered binary stdout and stderr; also PYTHONUNBUFFERED=x
see man page for details on internal buffering relating to '-u'
If the heart of this issue is to use pytest in a vim plugin, I think making the plugin set the environment variable before calling the entry point is perfectly acceptable.
Like @RonnyPfannschmidt I also do not want to change the default behavior as we can't know the repercussions of such a fundamental change, and introducing an option to set this seems like overkill when python already provides two ways for users to change that setting.
nicoddemus
on 20 Jan 2017
Closing it then.
Thanks!
blueyed
on 20 Jan 2017
I am using py.test version 2.4.2 and Python version 2.6.6 and want to view ANSI colours within less. These are some of the approaches which do not work
export PYTHONUNBUFFERED=1
py.test |& less -r
python -u -m pytest |& less -r
stdbuf --error=0 py.test | less -r
stdbuf --error=0 python -m pytest | less -r
What does work for me is
sudo yum install -y expect
unbuffer py.test |& less -r
lesgrieve
on 25 Sep 2019
export PYTHONUNBUFFERED=1 + py.test my_test.py -s works for me.
-s == 'disable all capturing'
itayB
on 3 Nov 2019
Related issues
thakkardharmik
·
3Comments
reaperhulk
·
3Comments
dmtucker
·
3Comments
nschloe
·
3Comments
jtmoon79
·
3Comments
Most helpful comment
Please also note that python supports setting this through an
PYTHONUNBUFFEREDenvironment variable:If the heart of this issue is to use pytest in a vim plugin, I think making the plugin set the environment variable before calling the entry point is perfectly acceptable.
Like @RonnyPfannschmidt I also do not want to change the default behavior as we can't know the repercussions of such a fundamental change, and introducing an option to set this seems like overkill when python already provides two ways for users to change that setting.