Pytest: Option to suppress exit code 5 (no tests run)

[jakirkham]

For the most part, I think the addition of PR ( https://github.com/pytest-dev/pytest/pull/817 ) to have a special exit code to denote no tests run was a good idea. However when I start out a project, I may not have any tests as there is no real code. Still I would like to get everything up and running (i.e. CIs). It would be nice in these cases like this to have an option to suppress this behavior and have exit code 0 returned instead.

[The-Compiler]

FWIW you can do something like

def test_placeholder():
    pass

[RonnyPfannschmidt]

im closing this, as the very reason for having and not suppressing this error code is to learn of broken/gone bad ci setups, its just as easy to simply put a first green test you know you will fill out later than to opt out of something you may forget about

[nicoddemus]

Thanks for the suggestion @jakirkham, but I agree with @RonnyPfannschmidt that we shouldn't add yet another option to pytest to handle this case specially because an alternative solution is trivial and easy to do, while the possible drawbacks can be a pain (forgetting later that the option is set like @RonnyPfannschmidt mentioned).

[mkleina]

But what in case the -m (markers) option is present and provided marker is not found? I am working on the case where multiple test suites are run in particular order using tox.

[testenv:all]
commands =
    pytest testsuite1/ {posargs}
    pytest testsuite2/ {posargs}
    pytest testsuite3/ {posargs}

where {posargs} are arguments passed to tox. When I want to preserve order, but filter out selected marker, for example:

tox -e all -- -m smoketest

pytest command fails when no markers found and no further tests are started because of exit code 5.

Any idea how to substitute this use case?

[blueyed]

@mkleina
You can wrap the command:
sh -c 'pytest testsuite1/ {posargs}; ret=$?; [ $ret = 5 ] && exit 0 || exit $ret'

I am using the following p wrapper myself:

#!/bin/sh
pytest "$@"
ret=$?
if [ "$ret" = 5 ]; then
  echo "No tests collected.  Exiting with 0 (instead of 5)."
  exit 0
fi
exit "$ret"

[RonnyPfannschmidt]

we ought to have a pytest plugin to do that for people that truely need it (since a shell wrapper is pretty anti-portable

[nicoddemus]

Agree with the plugin idea, should be simple to do.

[ChiKenNeg]

Any news?

[RonnyPfannschmidt]

@ChiKenNeg nobody worked on a plugin for this as far as i know, if you'd like to have one, i strongly suggest making one

[yashtodi94]

A little hack. But something on these lines should work fine. (Pytest v4.0.2)

Add this to your conftest.py:

def pytest_sessionfinish(session, exitstatus): if exitstatus == 5: session.exitstatus = 10 # Any arbitrary custom status you want to return

[Tadaboody]

@yashtodi94 would you be willing to create a plugin with that solution?

[yashtodi94]

Sure...I'll try submitting a PR by next week

[Tadaboody]

@yashtodi94 I think implementing the plugin in a different repository and linking it here is the norm, because it isn't changing pytest directly.
There's a cookie cutter template for making one that's easy to submit to pytest-dev
https://github.com/pytest-dev/cookiecutter-pytest-plugin
If you need any help let me know!

[yashtodi94]

@yashtodi94 I think implementing the plugin in a different repository and linking it here is the norm, because it isn't changing pytest directly.
There's a cookie cutter template for making one that's easy to submit to pytest-dev
https://github.com/pytest-dev/cookiecutter-pytest-plugin
If you need any help let me know!

Yes, I discovered that after posting my previous comment. Thanks for the help :)

[yashtodi94]

@Tadaboody @RonnyPfannschmidt

First attempt at building a plugin: https://pypi.org/project/pytest-custom-exit-code/

[nicoddemus]

Awsome @yashtodi94, thanks for sharing! 🙃