Tox: passenv: include TERM by default

[blueyed]

I think it would be good to include TERM in passenv by default, since it gets used with readline(-alike) programs, e.g. pdbpp (via pyrepl), where setupterm() then would fail without it being known.

Adding TERM to passenv explicitly works, but should not really be required, since there is usually no harm being done by passing this through, except for providing information about the used terminal.

doc ref: https://github.com/tox-dev/tox/blame/1883946bd3f233f58928864700eb0e6007332ffd/docs/config.rst#L388-L392
Related issue: https://github.com/tox-dev/tox/issues/250

Affected programs:

• Sphinx, uses no color without a decent TERM

[aarchiba]

Some tests will use the presence of TERM to detect that they're talking to a terminal and adapt their output to the terminal width, breaking doctests.

[blueyed]

@aarchiba
Thanks for your feedback. Do you have an example? (is this with pytest?)
Wouldn't those tests break when run outside of tox then also already?

[blueyed]

Some tests will use the presence of TERM to detect that they're talking to a terminal

I think usually isatty() is used for this.
$TERM is just an indicator to what a terminal is capable of (terminfo), but not that there is one attached necessarily.

[BenjamenMeyer]

Having it unset shouldn't be a problem - that should probably be the default.

If you are needing it set, then you may need to have different environments with it set to different values (linux, xterm, xterm-256color, vt100, etc) and want to make sure things are sane. IOW, when you care about it you're not going to want the user environment to potentially break your tests simply because the environment changed in an unexpected way.

FYI - I usually keep mine set to linux for various key controls; most systems default to xterm or xterm-256color; many servers however default to vt100.

[blueyed]

Having it unset shouldn't be a problem - that should probably be the default.

It is a problem when e.g. using pdb/pdb++ or anything else that might want to interact with the terminal. Even simple things like escape codes for cursor keys differ across terminals, and TERM is meant to provide the information to get the correct codes.

My point is that it shouldn't be a problem if it is set / passed through (usually).

If you are needing it set, then you may need to have different environments with it set to different values (linux, xterm, xterm-256color, vt100, etc) and want to make sure things are sane.

That would be the case when explicitly testing terminal apps, but then you would usually set this via pytest params etc for different scenarios anyway.

IOW, when you care about it you're not going to want the user environment to potentially break your tests simply because the environment changed in an unexpected way.

When you care about it you are setting / handling it already.

FYI - I usually keep mine set to linux for various key controls; most systems default to xterm or xterm-256color; many servers however default to vt100.

?
You mean outside of tox, in general?
You should have it set to the terminal (emulator) you are using. E.g. xterm-kitty, rxvt-unicode-256color, or tmux-256color etc. Things like ncurses, tput, infocmp etc use it to look up terminal capabilities. By using 'linux'/'vt100' always, you are not using features your terminal provides with e.g. Neovim.
Therefore it also does not really make sense to have e.g. TERM=xterm as a default (FWIW many terminals use TERM=xterm-256color by default, but recently it was updated with some capability for the real xterm, which they do not provide - resulting in scroll artifacts in Neovim then.)
Also, with ssh it makes sense to send it (given the terminfo is installed on the remote server).

[BenjamenMeyer]

@blueyed I wouldn't expect to be running a tox and then be dumped into pdb - that would actually break most CI environments where tox is meant to run, so it's outside the normal use-case.

If you are testing something that does require it then you would want to control it, not be user environment dependent. There are many applications (APIs) that tox is used for testing where you don't want or care if it is set, and as others have said having it set may make some tests produce undesired output. It is far easier to set it when needed than the other way around, so the default of having it unset is actually good.

And in fact you cannot guarantee it would be set on any CI system either ( neither Travis-CI https://docs.travis-ci.com/user/environment-variables/, and Circle-CI https://circleci.com/docs/2.0/env-vars/#built-in-environment-variables list is as a provided default environment variable; I'm sure you'll find similarly with other CI systems). Point being, by having it unset by default you match what your CI system will be like.

So if you are testing Vim or Vi, Emacs or other things that care about the TERM value, you'll probably also have separate environments for each supported TERM entry that you care to test - though honestly you'll probably have some other tool that will more easily automate having the many different TERM types.

Travis CI Documentation

Environment Variables

Using Environment Variables - CircleCI

A list of supported environment variables in CircleCI 2.0

[ntninja]

@BenjamenMeyer: mypy will not use colored terminal output, unless it can initialize curses to ask for the relevant terminal escape codes: https://github.com/python/mypy/blob/2f72de99f4b502be99f0781470969539efafc703/mypy/util.py#L525-L551

Unless TERM is passed down from the environment this will fail and running mypy from tox will display everything in plain text for no good reason. Note that mypy is not an interactive tool, but many CI environments (and good old users running tests) will benefit from the colored output nonetheless.

GitHub

python/mypy

Optional static typing for Python 3 and 2 (PEP 484) - python/mypy

[blueyed]

@BenjamenMeyer

I wouldn't expect to be running a tox and then be dumped into pdb - that would actually break most CI environments where tox is meant to run, so it's outside the normal use-case.

That's for a local use case, where you add the pdb.set_trace() yourself. Sure, you can also change tox.ini then, but it could be avoided.

as others have said having it set may make some tests produce undesired output

Only @aarchiba said this, and it sounded like it was confusing being connected to a terminal with having TERM set.
If it really was about "detect that they're talking to a terminal" then tox should rather close stdin for example.

I have not checked, but I would guess that Travis (and at least most Linux CIs) actually will have "$TERM" set. Otherwise apparently mypy would not display colors there.

[gaborbernat]

For me it feels that TERM is use case dependent if useful. If someone hits such a use case (e.g. mypy) just pass it through by specifying TERM within passenv.

[BenjamenMeyer]

@blueyed @alexander255 there's nothing stopping you from using the SETENV section of the tox config to set the TERM variable to achieve your goals. I'm just saying it shouldn't be a setting inherited from the user environment and should have a default of not being there. It's easy enough to setup the environment how you like:

[testenv]
setenv = 
    TERM=vt100

@alexander255 I linked to Travis-CI's and Circle CI's list of environment variables. They don't set TERM at all, or at least they don't specify that TERM is set so you cannot guarantee it will be set or set to anything sane. For consistency purposes it makes sense for them to _not_ set it.

[blueyed]

Sure, I know about setenv, and added it to 2+ projects already.
This was about having better defaults / a better user experience with tox by default.

@BenjamenMeyer
Not sure why I would use TERM=vt100 then though, and not pass through the user's.
To me it still seems there is a misunderstanding of what $TERM is used for.

Travis outputs terminal escape sequences all over (for colors etc) - it certainly has TERM set also.
FWIW you would not find $PATH, or $HOME in the list of documented env variables also I guess.

[ntninja]

@gaborbernat: Why should every person wanting colored output from mypy have to figure that a) mypy will suppress it's colored output if it cannot initialize curses, b) that initializing curses requires the TERM variable being set and c) that this requires setting passenv = TERM in the relevant toxenv section of the tox.ini file used?

@BenjamenMeyer: If Travis, Circle and however doesn't set the TERM variable then that means they do not declare themselves compatible to any existing terminal standard and hence not emitting any escape codes is actually the right thing to do. In practice I know of course that all modern terminals (except for the Windows Prompt*) are vt100 compatible in some way or another, so tools like pytest & tox can get away with just printing vt100-compatible escape codes if .isatty() is True and hope for the best.

As such hard-coding TERM=vt100 seems wrong and, if tox dev doesn't change their mind, setting passenv = TERM explicitely is the better approach. If you find that Travis, Circle or whatever fails to set this, then its better to set TERM=vt100 in their env-key instead. (And maybe bug their support to properly declare what terminal implementation is actually compatible with.)

Starting with some release of Windows 10 the *opt-in facility of having the Windows Prompt interpret vt100-compatible escape codes has been added.

[BenjamenMeyer]

@alexander255 perhaps the best solution to your issue is documentation - either here or in mypy. Not everyone needs TERM set or wants it set, and there's plenty of use-cases requiring it to be as it currently is. If it was really necessary then it would have gotten set a long time ago. $0.02 FWIW.

[blueyed]

there's plenty of use-cases requiring it to be as it currently is

Can you name one then, please? (This is what I've asked for 12 days ago basically.)
Even (or maybe especially?) pyrepl uses "passenv = TERM", although it is a readline-alike library that uses terminal information - but then sets this for tests explicitly.

If it was really necessary then it would have gotten set a long time ago. $0.02 FWIW.

It's not really necessary (nobody said that), since there is a workaround - and usually things that are not good terminal citizens (or do not need to be, i.e. they do not use info from TERM, but just basic colors etc (vt100)) are fine.
However it certainly does not help if tools block those "good citizens" by hiding necessary info.
If e.g. pytest would use curses for colors (without any fallback), then more people would have mentioned missing colors by now.

I clearly see the benefit of having setenv/passenv.
I can also see that it might actually change things also of course (but am curious about some example still), but mostly for the better (user experience) - and that is what defaults should be for.
(However there appears to be no delenv/unsetenv, is there?)

Anyway, at least I've learnt from this that you can have TOX_TESTENV_PASSENV=TERM locally at least.. o/

[BenjamenMeyer]

@blueyed there was an example given very early in this discussion - https://github.com/tox-dev/tox/issues/1441#issuecomment-544165413. I've never seen any kind of issue with tools under tox generating output that is user visible using the current default behavior (nose, nose2, pytest, pep8, pycodestyle, sphinx, bandit, flake8, black) as related to TERM. Out of all the projects I've worked on and interacted with, I'd say a project that is dependent on the value of TERM - the use-case this was opened for - is the exception, not the rule.

Typically in shell scripts (Bash, Batch, etc) you unset by setting the value equal to nothing; don't know if that would work with tox or not.

[blueyed]

https://github.com/tox-dev/tox/issues/1441#issuecomment-544165413 is not a concrete example, and I got the feeling that there is a misunderstanding - "talking to a terminal" means that fd 0, 1, or 2 are connected to a tty. Checking if "TERM" is set means knowing what terminal you are talking to. That's what I've said back then already.
So this still remains unanswered.

I've never seen any kind of issue with tools under tox generating output that is user visible using the current default behavior (nose, nose2, pytest, pep8, pycodestyle, sphinx, bandit, flake8, black) as related to TERM. Out of all the projects I've worked on and interacted with, I'd say a project that is dependent on the value of TERM - the use-case this was opened for - is the exception, not the rule.

Your examples are all "bad citizens" with regard to using terminal escape sequences: if they use colors they are just assuming the escape codes are being understood.
That's common and therefore likely fine in general, but tox is preventing "good citizens" access to information they need.

And as far as I can see it's just missing knowledge/laziness that e.g. pytest does not use curses, but hard-coded escape sequences.

Typically in shell scripts (Bash, Batch, etc) you unset by setting the value equal to nothing

Nope. Only Windows has this speciality (depending on the API you are using).
In Unix you would use unset, or env -u TERM ….
Typically unset is different from empty.

% env -u TERM sh -c 'echo $TERM'
dumb
% env TERM= bash -c 'echo $TERM'

(you can see here that bash defaults to TERM=dumb if unset, same for dash (more POSIX))

Given this example, what about tox using TERM=dumb by default?
This would avoid the issue with curses.setupterm() failing, although still results in no colors with mypy, since dumb terminals do not have them:

% TERM=dumb infocmp
#   Reconstructed via infocmp from file: /usr/share/terminfo/d/dumb
dumb|80-column dumb tty,
    am,
    cols#80,
    bel=^G, cr=\r, cud1=\n, ind=\n,

; don't know if that would work with tox or not.

Yeah, the question was if this could easily be unset in tox.
It could by prefixing your command with env -u, but that requires to whitelist env then also.

[blueyed]

Just for completeness: this is how mypy initializes colors: https://github.com/python/mypy/blob/95a41fa1a92576f94ae36ee658fbe4dd36a00ef7/mypy/util.py#L525-L551
I.e. for TERM=dumb it will not use colors, while pytest (via py) does this:
https://github.com/pytest-dev/py/blob/34f716fe1fb4df9a5257b5f640ebb8a71e10aa88/py/_io/terminalwriter.py#L173-L187

Note that it will also not use colors for TERM=dumb, but not because it uses the actual terminfo DB, but has the name hardcoded (as with the color codes)!

So if you copy "dumb" to "dumb2" it will use (hardcoded) colors with "TERM=dumb2", making it a "bad citizen" (had a longer comment here, but the bot killed it by changing my comment).
Resulting in issues like https://github.com/pytest-dev/pytest/issues/4460.

GitHub

python/mypy

Optional static typing for Python 3 and 2 (PEP 484) - python/mypy

GitHub

pytest-dev/py

Python development support library (note: maintenance only) - pytest-dev/py

[blueyed]

@BenjamenMeyer
So given the TERM=dumb case: since you (and @aarchiba from what I understood) are eager to have an isolated env, it should probably default to "dumb" then.
How would you react to pytest users missing colors then? Refer them to the doc/FAQ?

[BenjamenMeyer]

@blueyed I would leave it as it is - default it to not set TERM at all.

Whether or not pytest has colors doesn't bother me. If you take into account folks that are color-blind - which varies all over from not seeing some colors to any color - using coloration could actually be problematic for some.

Per tox, I would simply state in the docs/FAQ what the default is. Tools like pytest, mypy, etc if they need a specific setting to get something working then they should be responsible for documenting that in their docs/FAQ.

[blueyed]

Ok, well - I'm still curious about a concrete example where this makes a difference for the worse.

Per tox, I would simply state in the docs/FAQ what the default is.

They are documented.

Tools like pytest, mypy, etc if they need a specific setting to get something working then they should be responsible for documenting that in their docs/FAQ.

The point here is that it is common sense that "TERM" is set within terminals (and tox runs in a terminal).

btw: tox passes through LANG/LANGUAGE, which actually makes a difference often - so much that you often use LANG=C for testing to not use any translations.
https://github.com/tox-dev/tox/blob/e7136f55c98628688725986a4dd2141df2a02ef2/src/tox/config/__init__.py#L665-L674

[BenjamenMeyer]

@blueyed said:

I'm still curious about a concrete example where this makes a difference for the worse.

If you change it, you'll find out since the cases where it makes a difference will then break since it would be a breaking change from historic behavior.

Though as you noted the unset is not available, so if you do change it, also introduce unset so there's an easy remediation path, and that might be a good use for the LANG case you mention as well.

That said, it was earlier said:

That's common and therefore likely fine in general, but tox is preventing "good citizens" access to information they need.

No, tox is not preventing it. It's just making it an explicit requirement to specify it if needed or desired.

Per the analysis of mypy vs pytest in https://github.com/tox-dev/tox/issues/1441#issuecomment-548683479 - interesting analysis. I wouldn't _necessarily_ call pytest a bad citizen since they're not using curses and provide a configuration for it to turn it on/off. Perhaps their default should be different. Not all platforms use TERM so it make sense for what they do.

[blueyed]

@BenjamenMeyer
Cool, so we've gone from "it has to be like that" to "you will find out".
Apparently nobody has an example at hand.. :man_shrugging:

I wouldn't _necessarily_ call pytest a bad citizen since they're not using curses and provide a configuration for it to turn it on/off. Perhaps their default should be different. Not all platforms use TERM so it make sense for what they do.

Well, if the terminfo says "I can not do colors", a tool should not send "weird characters" (escape codes to use colors, not understood by the terminal) - don't you think?

"good citizens" in that sense are those that go the extra mile to use the terminfo, not just assume something. It is like using tput setaf 1 to get the sequences for red foreground in contrast to just assume it will be ^[[31m (note that with rxvt-unicode/urxvt it is ^[[38;5;1m already, but I think it just also handles the more common case then).

Also note that TERM=vt100 (since you've mentioned to use this even by default?) actually appears to have no colors in that sense? (TERM=vt100 tput setaf 1 outputs nothing.)
I.e. pytest outputting colors there is wrong, and it only works for you since your terminal then understands the color codes it uses.

While I would still suggest to pass through TERM (since it helps with getting the expected behavior, especially when not using TERM=dumb (which would be my less preferred suggestion)), I've put enough time into this for now I think.. :D

[blueyed]

Apropos handling TERM=dumb: https://github.com/tox-dev/tox/issues/1290 ;) (only came across it when searching for this issue here)

[BenjamenMeyer]

@blueyed #1290 I believe would be a different issue as that is regarding tox's behavior regarding it's own operation in the general environment and not necessarily the behavior of applications running within the generated tox virtual environments - the screen shot shown and complaint isn't even about the virtual environments.

[BenjamenMeyer]

Cool, so we've gone from "it has to be like that" to "you will find out".

My point was more that there are plenty of use-cases out there you won't necessarily know about until changing the default behavior and have them come out of the woodworks complaining. Some may just fix stuff (if it's easy enough to do), others may complain. Hopefully the community as a whole would be as pleasant as this discussion has been if such a change does happen.

Well, if the terminfo says "I can not do colors", a tool should not send "weird characters" (escape codes to use colors, not understood by the terminal) - don't you think?

If terminfo is available - again not all platforms or shells support terminfo - and the tool understands terminfo, then yes. I think most devs don't really understand terminfo and just do what works for them until someone comes along and tells them otherwise, or contributes such functionality.

NOTE: MS dropped support for ANSI term info, and are really only adding it back now due to the whole Windows Subsystem For Linux thing (see https://en.wikipedia.org/wiki/ANSI_escape_code). DOS and Command shell (command.com, cmd.exe, PowerShell) were never really TTY device backed.

The default of not having TERM provides the most flexibility since users can specify it if desired.

I've put enough time into this for now I think.. :D

Likewise. Also, thanks for the civil discussion!

ANSI escape code - Wikipedia

[blueyed]

For reference: Sphinx is also affected by this: https://github.com/sphinx-doc/sphinx/pull/7248

[blueyed]

..which means basically any Python project using Sphinx with a "docs" testenv (in case that was not clear).