Pytest: add pytest.mark.usefixtures mark in a hook not effective

[diefans]

• [y] a detailed description of the bug or suggestion
• [-] output of pip list from the virtual environment you are using
• [y] pytest and operating system versions: archlinux, python 3.8.1, pytest 4.6.9
• [y] minimal example if possible

I had the naive idea to apply additional fixtures on behalf of a custom marker. Unfortunately the usefixture marks are set, but not applied. So I wonder if this is a bug, or even if there is the right way to do this, if possible.

dir=test_apply_usefixtures
mkdir -p $dir

(
cd $dir
touch __init__.py

cat << EOF > conftest.py
import pytest

CUSTOM_MARK = "foobar"


def pytest_configure(config):
    config.addinivalue_line("markers", f"{CUSTOM_MARK}(*args): marker")


def pytest_collection_modifyitems(items):
    for item in items:
        if (
            CUSTOM_MARK in set(mark.name for mark in item.iter_markers())
            and "prepare_something" not in item.fixturenames
        ):
            item.add_marker(pytest.mark.usefixtures("prepare_something"))


@pytest.fixture
def prepare_something(request, mocker):
    from . import something

    stuff = next(
        (mark.args for mark in request.node.iter_markers() if mark.name == CUSTOM_MARK),
        (),
    )
    mocker.patch.object(something.Foo, "stuff", set(stuff))
EOF

cat << EOF > something.py
class Foo:
    stuff = set()
EOF

cat << EOF > test_foo.py
import pytest


@pytest.mark.foobar("foo", "bar")
def test_foo_fails(request):
    assert request.node.own_markers == [
        pytest.mark.foobar("foo", "bar").mark,
        pytest.mark.usefixtures("prepare_something").mark,
    ]
    from . import something

    assert something.Foo.stuff == {"foo", "bar"}


@pytest.mark.foobar("foo", "bar")
@pytest.mark.usefixtures("prepare_something")
def test_foo_works(request):
    assert request.node.own_markers == [
        pytest.mark.usefixtures("prepare_something").mark,
        pytest.mark.foobar("foo", "bar").mark,
    ]
    from . import something

    assert something.Foo.stuff == {"foo", "bar"}
EOF
)
pytest $dir

========================================== test session starts ==========================================
platform linux -- Python 3.8.1, pytest-4.6.9, py-1.8.1, pluggy-0.13.1
Using --randomly-seed=1584054304
benchmark: 3.2.3 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/foobar, inifile: pytest.ini
plugins: doctestplus-0.5.0, randomly-3.2.1, mock-2.0.0, asyncio-0.10.0, benchmark-3.2.3, cov-2.8.1
collected 2 items

test_apply_usefixtures/test_foo.py .F                                                             [100%]

=============================================== FAILURES ================================================
____________________________________________ test_foo_fails _____________________________________________

request = <FixtureRequest for <Function test_foo_fails>>

    @pytest.mark.foobar("foo", "bar")
    def test_foo_fails(request):
        assert request.node.own_markers == [
            pytest.mark.foobar("foo", "bar").mark,
            pytest.mark.usefixtures("prepare_something").mark,
        ]
        from . import something

>       assert something.Foo.stuff == {"foo", "bar"}
E       AssertionError: assert set() == {'bar', 'foo'}
E         Extra items in the right set:
E         'bar'
E         'foo'
E         Use -v to get the full diff

test_apply_usefixtures/test_foo.py:12: AssertionError
======================================= slowest 10 test durations =======================================
0.03s setup    test_apply_usefixtures/test_foo.py::test_foo_works

(0.00 durations hidden.  Use -vv to show these durations.)
======================================== short test summary info ========================================
FAILED test_apply_usefixtures/test_foo.py::test_foo_fails - AssertionError: assert set() == {'bar', 'f...
================================== 1 failed, 1 passed in 0.08 seconds ===================================

[diefans]

After digging into https://github.com/pytest-dev/pytest-asyncio/blob/master/pytest_asyncio/plugin.py to get some inspiration, the solution to this problem is quite simple:

def pytest_runtest_setup(item):
    if CUSTOM_MARK in item.keywords and "prepare_something" not in item.fixturenames:
        item.fixturenames.append("prepare_something")

[diefans]

PS: I was not able to find a reasonable documentation for fixturenames

[antonlisovenko]

@diefans thanks man, spent half a day on trying to make pytest_collection_modifyitems + usefixtures work and didn't manage to do it, but the pytest_runtest_setup did a super trick!
(Very confusing that it didn't work - I'd regard this as a bug tbh and would recommend you to reopen the issue back)

[diefans]

as @antonlisovenko suggests and since it is either a documentation issue or a bug I reopen this issue...

[nicoddemus]

It is not a bug, but by design. We have this note in the introduction for marks:

Any hints on how to update the docs for that to be clearer?

[nicoddemus]

Closing this again and opened #7422 about the docs. 👍

[nicoddemus]

Btw, thanks @diefans for the diligence of following up, we appreciate it. 😁

[diefans]

My innocent idea was to use item.add_marker(pytest.mark.usefixtures("prepare_something")) to add a mark resp. a fixture to a test via another mark... At least the term "add_marker" made me believe, that's what I want. The fixture mark is also present, but without impact/effect.

So to document this, one has to mention:

1. that add_marker(usefixtures("foobar")) is not loading the fixture
2. to load/apply a fixture via another mark, one has to go the above way item.fixturenames.append