Pytest: Way to change name of skipped tests

[davidak]

I have modified the tests to use the docstring as name, as described on stackoverflow, but it don't work for skipped tests.

========================================= short test summary info ==========================================
SKIPPED [1] /home/admin/gsm-verification/tests/test_debian.py:41: No memory size in gsm settings
SKIPPED [1] /home/admin/gsm-verification/tests/test_debian.py:56: No CPU core number in gsm settings
✘ Check for errors in system log
✘ Check GOS version using gsmctl
✘ Check GOS version using lsb_release -r
✘ Check LSB description using lsb_release -d
✔ Check for correct installation of all packages
✔ Check build number
✔ Check if service is running

Is there a way that it also applies to skipped tests or is the behavior hardcoded?

I would like to have it this way:

SKIPPED Check memory size (No memory size in gsm settings)
SKIPPED Check number of CPU cores (No CPU core number in gsm settings)
✘ Check for errors in system log
✘ Check GOS version using gsmctl

(Why use the path and test line in general? You are not able to tell what test it is without the name!)

My conftest.py:

def pytest_itemcollected(item):
    # Use docstring instead of function name when available for test items
    node = item.obj
    item._nodeid = node.__doc__.strip() if node.__doc__ else node.__name__

---

operating system: GOS 4.3.5 (based on debian stretch)
python: 3.5.3
pytest: 4.2.0

pip list

root@gsm:/home/admin/gsm-verification# pip3 list
DEPRECATION: The default format will switch to columns in the future. You can use --format=(legacy|columns) (or define a format=(legacy|columns) in your pip.conf under the [list] section) to disable this warning.
astroid (2.1.0)
atomicwrites (1.2.1)
attrs (18.2.0)
cryptography (1.7.1)
defusedxml (0.4.1)
Feed (0.39)
fume (1.2)
gos-state-manager (1.0)
gsm-upgrade (0.0.0)
gsmctl (3.0.1)
gvm-tools (1.4.1)
idna (2.2)
isort (4.3.4)
Jinja2 (2.8)
keyring (10.1)
keyrings.alt (1.3)
lazy-object-proxy (1.3.1)
lxml (3.7.1)
MarkupSafe (0.23)
mccabe (0.6.1)
more-itertools (5.0.0)
netifaces (0.10.4)
ntplib (0.3.3)
paramiko (2.0.0)
pathlib2 (2.3.3)
pip (9.0.1)
pluggy (0.8.1)
py (1.7.0)
pyasn1 (0.1.9)
pycrypto (2.6.1)
pygobject (3.22.0)
pylint (2.2.2)
PySpatch (2.5)
pytest (4.2.0)
pytest-custom-report (1.0.0)
python-dateutil (2.5.3)
pythondialog (3.4.0)
pyxdg (0.25)
PyYAML (3.12)
SecretStorage (2.3.1)
selfcheck (1.1)
setuptools (33.1.1)
Shepherd (0.2.30)
six (1.10.0)
typed-ast (1.2.0)
Werkzeug (0.11.15)
wheel (0.29.0)
wrapt (1.11.1)

[nicoddemus]

Hi @davidak,

Probably we should show the node id instead of path/line number:

SKIPPED [1] tests/test_debian.py::test_gsm_memory: No memory size in gsm settings

I don't think showing just the reason is the better option, because it is not obvious which test actually skipped.

What do you think?

[davidak]

Probably we should show the node id instead of path/line number

Yes, that would be good. In my case the nodeid would be the docstring, like you see in my example.

I don't think showing just the reason is the better option

Yes, i didn't suggested that. My output actually contains the test description.

SKIPPED Check memory size (No memory size in gsm settings)

SKIPPED = Status
"Check memory size" = nodeid (from docstring)
"No memory size in gsm settings" = Skip reason

[nicoddemus]

Oh I see, I missed the detail of your conftest customization. 👍

Instead of hacking the internal _nodeid name, we can possibly can make that part of the actual functionality by introducing a new hook which gets the "test description", which by default uses function.__doc__ or something like this.

[RonnyPfannschmidt]

there is pytest plugins that do this style of output correctly, this should be experimented on with those

[nicoddemus]

Oh right, including @wimglenn's https://pypi.org/project/pytest-custom-report.

[davidak]

@RonnyPfannschmidt i was not able to modify it with conftest.py.

Can you give me a hint how the _nodeid variable of skipped tests is called?

I think it would make sense to have consistent output here with the other tests.

[RonnyPfannschmidt]

@davidak it should absolutely be avoided to modify _nodeid as it completely throws internal consistency aboard - do NOT do it

instead take an example on pytest plugins like pytest-sugar, which turn off normal pytest output and do entirely custom output - i recall there was a plugin which does output very close to what you want but i dont remember the name

[davidak]

@RonnyPfannschmidt you probably mean pytest-pspec, but i thought 3 lines of code are simpler than an additional dependency.

Changing the _nodeid is the most upvoted answer here, so it seems to work for many people. https://stackoverflow.com/a/39035226/2611995

[RonnyPfannschmidt]

@davidak i absolutely don't care how popular that hack is, i'm telling you its severely broken

there will be surprise breakages, and we will change that internal api in future

[CarycaKatarzyna]

As far as I understand this issue is just a discussion about the output's personalization and how dangerous it is. Is there anything to implement as a solution to this issue? If not, I suggest closing this problem or at least changing the label to "question" because the current labels are misleading.