Pytest: No colors by default on GitHub Actions

[The-Compiler]

When running pytest on GitHub Actions (like we do ourselves, random example), the output is not colored.

pytest does color output on Travis CI by default, so it could be argued that this is somehow GitHub Action's fault. However, other tools (e.g. eslint, and I think mypy) seem to colorize their output on GitHub actions just fine, without needing to force them! From what I've seen, those tools seem to be influenced by the TERM environment variable, while pytest's (or pylib's?) autodetection seems to work differently?

As a workaround, this helps (as part of the step definition):

      - name: "Run pytest"
        env:
          PYTEST_ADDOPTS: "--color=yes"
        run: pytest

(If you run pytest via tox, make sure to add PYTEST_ADDOPTS to passenv!)

[hugovk]

Yep, this happens with pre-commit too, and many, many other tools. The problem is GitHub Actions does not provide a tty: https://github.com/actions/runner/issues/241. This issue mentions pytest, and GitHub say it's low in their backlog and they have no plans to fix it (https://github.com/actions/runner/issues/241#issuecomment-580480162).

If you run pytest via tox, a similar workaround is to put it in config (eg. setup.cfg) so you don't need passenv:

• https://github.com/python-pillow/Pillow/pull/4716/files

[hugovk]

Please see PR https://github.com/pytest-dev/pytest/pull/7462.

[eine]

IMHO, envvars to force colouring are VERY inconsistent. This is true even in the Python ecosystem. For example, see this issue, tartley/colorama#230 (/cc @jhol @tartley @wiggin15 @eric-wieser), tox-dev/tox#1468 (/cc @gaborbernat)... I believe it would be desirable if the community could talk and agree on a couple of envvars.

I like @jhol's FORCE_COLOR and NO_COLOR.

[The-Compiler]

@eine True, but this seems unrelated to this issue?

[eine]

@The-Compiler, I believe it is! Should pytest work with FORCE_COLOR (and tox and colorama too), PYTEST_ADDOPTS or --color=yes would not be required. At the same time, if checking TERM or asking GitHub to define FORCE_COLOR is the way to go, all projects (including pytest) could benefit from it. #7462 is a legit workaround, but an ad-hoc fix after all.

EDIT

FTR:

• https://github.com/VUnit/vunit/blob/b34a4f3c341f2b4175db678ac7647c32ce48bfae/.github/workflows/push.yml#L12
• https://github.com/VUnit/vunit/blob/b34a4f3c341f2b4175db678ac7647c32ce48bfae/.github/workflows/push.yml#L52
• https://github.com/VUnit/vunit/runs/846846336?check_suite_focus=true#step:7:80

[jhol]

NO_COLOR is a community standard environement variable.

The general behavior is:

• A command line application emits colour when attached to a TTY, but no colour otherwise e.g. when attached to a pipe.
• NO_COLOR stops the application from emitting colour even when attached to a TTY
• My extension to the standard is to add FORCE_COLOR which forces the application to emit colour escape sequences even when not outputting to a TTY e.g. if we want to pipe into a log file with colour.

[nicoddemus]

Thanks @jhol and @eine for bringing this up. I've created #7464 to track adding this support to pytest. 👍

[nicoddemus]

I think based on the discussions on https://github.com/pytest-dev/pytest/pull/7462, we can close this as "wontfix" because pytest can't reliably detect this in GitHub actions. What do you think @The-Compiler ?

[eine]

Agree. As @The-Compiler commented in https://github.com/pytest-dev/pytest/pull/7462#issuecomment-656068713, I think that the following is the way to go:

env:
  PY_COLORS: "1"

For example: https://github.com/VUnit/vunit/blob/master/.github/workflows/push.yml#L12

When/if tox supports it:

env:
  FORCE_COLOR: "1"

[The-Compiler]

Sounds reasonable!