Pytest: How can autouse fixtures be used with regular modules that use doctests?

[NeilGirdhar]

import pytest
import numpy as np


@pytest.fixture(autouse=True)
def set_printoptions_fixture(doctest_namespace):
    np.set_printoptions(suppress=True, precision=5)


def sin():
    """
    >>> a = np.array([0.2])
    >>> np.sin(a)
    array([ 0.19867])
    """
    pass

Works fine when calling pytest on the file itself, e.g. pytest a.py, but if a.py is placed in a module abcd, and then pytest is called pytest abcd, this fails with DocTestFailure since the fixture is not run.

[nicoddemus]

but if a.py is placed in a module abcd

You mean without the .py extension? That is somewhat expected I think, fixtures will be collected from .py files only.

[NeilGirdhar]

by module, I mean a directory that contains an __init__.py:

~/src/imm: pytest color/a.py
==================================================================================== test session starts =====================================================================================
platform darwin -- Python 3.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /Users/neil/src/imm, inifile: pytest.ini
collected 1 items

color/a.py .

================================================================================== 1 passed in 0.26 seconds ==================================================================================
~/src/imm: pytest color/
==================================================================================== test session starts =====================================================================================
platform darwin -- Python 3.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0
rootdir: /Users/neil/src/imm, inifile: pytest.ini
collected 1 items

color/a.py F

========================================================================================== FAILURES ==========================================================================================
___________________________________________________________________________________ [doctest] color.a.sin ____________________________________________________________________________________
011
012     >>> a = np.array([0.2])
013     >>> np.sin(a)
Expected:
    array([ 0.19867])
Got:
    array([ 0.19866933])

/Users/neil/src/imm/color/a.py:13: DocTestFailure
================================================================================== 1 failed in 0.11 seconds ==================================================================================

[timofurrer]

by module, I mean a directory that contains an __init__.py:

That would be a package. A module is just a Python file ending with *.py.

I've analyzed this a little bit further:

It appears that the fixture is not run if the module is not following the naming conventions:

1. (fail) run pytest given the test directory name with module NOT following naming conventions:

$ pytest foo/ --doctest-modules                                                                                                                        [±master ●] [20:48:54]
=============================================================================================== test session starts ================================================================================================
platform linux -- Python 3.6.1, pytest-3.0.8.dev, py-1.4.33, pluggy-0.4.0
rootdir: /home/tuxtimo/work/pytest, inifile: tox.ini
collected 1 items 

foo/issue_2408.py F

===================================================================================================== FAILURES =====================================================================================================
___________________________________________________________________________________________ [doctest] foo.issue_2408.sin ___________________________________________________________________________________________
012 
013     >>> a = np.array([0.2])
014     >>> np.sin(a)
Expected:
    array([ 0.19867])
Got:
    array([ 0.19866933])

/home/tuxtimo/work/pytest/foo/issue_2408.py:14: DocTestFailure
============================================================================================= 1 failed in 0.06 seconds =============================================================================================

1. (pass) run pytest given the module name NOT following naming conventions*:

$ pytest foo/issue_2408.py --doctest-modules
=============================================================================================== test session starts ================================================================================================
platform linux -- Python 3.6.1, pytest-3.0.8.dev, py-1.4.33, pluggy-0.4.0
rootdir: /home/tuxtimo/work/pytest, inifile: tox.ini
collected 1 items 

foo/issue_2408.py .

============================================================================================= 1 passed in 0.07 seconds =============================================================================================

1. (pass) run pytest given the test directory name with module following naming conventions:

$ pytest foo/ --doctest-modules                                                                                                                        [±master ●] [20:51:02]
=============================================================================================== test session starts ================================================================================================
platform linux -- Python 3.6.1, pytest-3.0.8.dev, py-1.4.33, pluggy-0.4.0
rootdir: /home/tuxtimo/work/pytest, inifile: tox.ini
collected 1 items 

foo/test_issue_2408.py .

============================================================================================= 1 passed in 0.06 seconds =============================================================================================

1. (pass) run pytest given the module name following naming conventions:

$ pytest foo/test_issue_2408.py --doctest-modules                                                                                                      [±master ●] [20:51:05]
=============================================================================================== test session starts ================================================================================================
platform linux -- Python 3.6.1, pytest-3.0.8.dev, py-1.4.33, pluggy-0.4.0
rootdir: /home/tuxtimo/work/pytest, inifile: tox.ini
collected 1 items 

foo/test_issue_2408.py .

============================================================================================= 1 passed in 0.07 seconds =============================================================================================

[nicoddemus]

Thanks @timofurrer for investigating it further.

@NeilGirdhar let me explain what's happening: pytest collects fixtures from conftest.py files or test modules. Any file passed explicitly on the command-line is considered a test module, that's why the auto use fixture is active and the doc test can see it. This is by design, so pytest doesn't have to load every .py module in the search directories looking for fixtures and tests, which would slow down test discovery considerably.

When the file is moved inside a package without following the test module name convention, then the fixture is not collected that's why it is not active during the doc test.

So to summarize, I suggest that you move the auto use fixture to color/conftest.py. This way pytest will collect the fixture and enable it for the execution of the doc test.

[timofurrer]

@nicoddemus that makes sense. However, I'm wondering why not using the fixtures in modules which are not following the naming conventions but which are loaded anyways, like the one in this question?
Neither the module nor the test function itself follows the naming conventions. Is there are particular reason why it's still considered a test? If this is intended behavior wouldn't it make sense to check if the loaded module contains any fixtures as well?

[nicoddemus]

Usually test code is not mixed with production code, so this is usually not an issue. Fixtures from arbitrary modules are not part of the collection tree built by pytest (with contest files and test modules), so changing this would not be simple I think.

But keep in mind that we are not talking about normal tests, but doc tests in docstrings which are a more exotic case.

[NeilGirdhar]

Thanks for the explanation, everyone.

The issue here is that the doctests are written in regular modules. They are not factored out into a module that follows testing naming conventions. The problem with putting the fixture into conftest.py is that it would then apply to all modules in the package, right? But I want it to be specific to the module. I guess I could factor out the module into a subpackage, and put a conftest .py into that subpackage?

[nicoddemus]

But I want it to be specific to the module. I guess I could factor out the module into a subpackage, and put a conftest .py into that subpackage?

That's a solution, yes. 👍