Pytest: Allow having multiple conftest.py on same level

[ikonst]

In our project, the conftest.py files have grown so large that we felt the need to refactor them into smaller modules. However, modules imported by conftest.py using import cannot contribute fixtures.

We've tried using pytest_plugins but unfortunately (as the documentation says and as my testing confirms), importing a plugin with fixtures in one conftest.py makes them available in tests in parent and sibling hierarchies.

I'd like to propose a simple solution, to have pytest look for conftest.py and then conftest_*.py (under the same directory). This way, unruly conftest.py files could be easily split up.

(Let me know if it's an acceptable solution, I can implement it.)

[pytestbot]

GitMate.io thinks possibly related issues are https://github.com/pytest-dev/pytest/issues/3212 (Having all options from all conftest.py for all collected tests), https://github.com/pytest-dev/pytest/issues/1931 (Nested conftest.py), https://github.com/pytest-dev/pytest/issues/2532 (TypeError on conftest.py during collection), https://github.com/pytest-dev/pytest/issues/11 (documentation typo "conftest.py"), and https://github.com/pytest-dev/pytest/issues/1404 (conftest.py loaded in wrong order.).

[nicoddemus]

Hi @ikonst,

For your use case, where you want fixtures in separate files just as a means to keep things more manageable, you can import the fixtures normally into the conftest.py file:

# conftest.py
from ._app_fixture import app
from ._regression_fixture import regression

And so on. Importing fixtures into test files in general is not recommended as it might lead to subtle problems, but importing them into the conftest.py file (and nowhere else) looks like it will solve your problem.

[ikonst]

Would that force me to import fixtures one by one, though?

e.g. let's say I have 100 fixtures: 50 are data model fixtures (that I'd like to store in "conftest_data_models.py") and 50 are network call fixtures ("conftest_network_calls.py").

[nicoddemus]

No, you can import them using *; as long as they are in the module's namespace by the time pytest looks for fixtures, it will work.

[RonnyPfannschmidt]

@ikonst note that due to bugs pytest_plugins in conftests also breaks strangely

[blueyed]

I wonder if it would be feasible or possible already to have a module "conftest"?
I.e. conftest/__init__.py and several other files in conftest/, that get imported in __init__.py.

[nicoddemus]

Currently we look explicitly for conftest.py files:

https://github.com/pytest-dev/pytest/blob/4d0297b4138b694e1d99f4c8147aaf06d725d0f4/src/_pytest/config.py#L367-L370

As to change this to support packages, it is possible I guess, but it would mean to change other places that also look for conftest.py files, for example:

https://github.com/pytest-dev/pytest/blob/4d0297b4138b694e1d99f4c8147aaf06d725d0f4/src/_pytest/doctest.py#L354-L355

https://github.com/pytest-dev/pytest/blob/4d0297b4138b694e1d99f4c8147aaf06d725d0f4/src/_pytest/fixtures.py#L1054-L1057

Because of the possibility of introducing more bugs, I'm not sure it is worth trying to add support for this given that the "workaround" is pretty simple: import your symbols into the conftest.py file, which you would need to do anyway with the conftest/__init__.py approach.

[RonnyPfannschmidt]

im -1 on this

[nicoddemus]

Yep, I agree with @RonnyPfannschmidt here.

@ikonst did importing the fixtures into the conftest.py fixed your original issue?

[ikonst]

Thanks for all your help. I'll test this on my codebase and report back.

[nicoddemus]

Thanks!

[ikonst]

from .fixtures.models import * worked out great

[blueyed]

Great.
Closing this issue then.

[ikonst]

To follow up, my approach to import all fixtures under fixtures/* is adding to conftest.py:

from pkgutil import walk_packages

from . import fixtures

for package in walk_packages(fixtures.__path__, prefix=fixtures.__name__ + '.'):
    module = package.module_finder.find_module(package.name).load_module()
    globals().update({k: v for k, v in vars(module).items() if not k.startswith('_')})

Having to copy-paste this code snippet across multiple conftest.py files seems suboptimal, but so does implicitly specifying all fixtures, e.g.

from .fixtures.foo import *  # noqa
from .fixtures.bar import *  # noqa

and the # noqas are somewhat noisy.

Furthermore, I didn't find a way to write a pytest plugin to hook into loading of conftest.py files, in order to inject this functionality.