Pytest: Pytest adds parent instead of expected directory into python path. (shouldn't it add rootdir?)

[cepbrian]

Using pytest 3.1.3, python 2.7, OSX 10.12.5

Looking at https://docs.pytest.org/en/latest/customize.html Which appears to be the most recent documentation.

Documentation says that it looks for pytest.ini to find rootdir. Instead it looks up from tests dir until it finds a dir without __init__.py.

I'm not sure if this is a documentation error, a linking error in the docs (It's not pointing to the most recent version of the docs somehow) or an actual error in the code. I'm guessing it's just that the docs aren't up to date, but that's just a guess.

EDIT:

It appears that I asked the wrong question. (leaving it there for clarity)

The question isn't actually about rootdir. The issue is that the documentation doesn't clearly say what pytest puts in PYTHONPATH. As such, I assumed it put rootdir. And so I assumed it was not setting rootdir correctly. It turns out that it is setting rootdir just fine, but it doesn't use rootdir in pythonpath at all.

I believe that there should be some (perhaps just some easier to find) documentation on how pytest sets pythonpath.

[nicoddemus]

Hi @cepbrian thanks for writing.

A quick look at the code indicates that pytest first looks for pytest.ini, tox.ini and setup.cfg in getcfg, and if that fails it falls back to finding a setup.py file and then using the common ancestor of all arguments given in the command-line.

I might be missing something (it is late and I should be in bed by now), but can you provide an example where pytest ignores pytest.ini to determine the rootdir? If that's happening for you it might indeed be a bug.

[cepbrian]

(venv) turlingdrome$ pwd
/Users/brianp/work/testingtest

(venv) turlingdrome$ tree
.
├── __init__.py
├── __init__.pyc
├── pytest.ini
└── tests
    ├── __init__.py
    ├── __init__.pyc
    ├── __pycache__
    │   └── test_foo.cpython-27-PYTEST.pyc
    └── test_foo.py

2 directories, 7 files

(venv) turlingdrome$ pytest
============================= test session starts ==============================
platform darwin -- Python 2.7.10, pytest-3.1.3, py-1.4.34, pluggy-0.4.0
rootdir: /Users/brianp/work/testingtest, inifile: pytest.ini
collected 1 item s

tests/test_foo.py F

=================================== FAILURES ===================================
___________________________________ test_foo ___________________________________

    def test_foo():
        print os.getcwd()
        print sys.path
>       assert 1==0
E       assert 1 == 0

tests/test_foo.py:8: AssertionError
----------------------------- Captured stdout call -----------------------------
/Users/brianp/work/testingtest
['/Users/brianp/work', '/Users/brianp/work/cyan/venv/bin', '/Users/brianp/work/cyan/venv/lib/python27.zip', [ETC ETC ETC]]

Looking at this more carefully, the problem ISN'T rootdir. The problem is that rootdir isn't what's put into pythonpath.

[nicoddemus]

Hmm rootdir seems correct:

============================= test session starts ==============================
platform darwin -- Python 2.7.10, pytest-3.1.3, py-1.4.34, pluggy-0.4.0
rootdir: /Users/brianp/work/testingtest

given that /Users/brianp/work/testingtest contains a pytest.ini file. Perhaps you were under a different impression what rootdir means?

[cepbrian]

The question is how is rootdir used? shouldn't that be the thing that is put into pythonpath?

If not, then there should be some more documentation somewhere about how pytest determines its pythonpath, because it's doing something weird...

[cepbrian]

Pytest is clearly adding the work directory, because when I run with python -m pytest, I get:

['/Users/brianp/work', '', '/Users/brianp/work/cyan/venv/lib/python27.zip',

python adds '' as normal. and then pytest adds /Users/brianp/work...

[nicoddemus]

The question is how is rootdir used? shouldn't that be the thing that is put into pythonpath?

OK that's a problem people often run into. I updated the title of the issue to better reflect that.

The rootdir is used to construct the nodeid of the test items, not to set the PYTHONPATH, take a look at the complete explanation here.

Pytest is clearly adding the work directory, because when I run with python -m pytest

That happens because your test is in a package (in the same directory as an __init__.py file), and pytest will try to find its top-level package by looking up in the hierarchy until it finds a folder without a __init__.py file, in which case it assumes that's the toplevel package. In other words because there's a __init__.py file next to test_foo.py, pytest will try to import it as turlingdrome.tests.test_foo, and to do that it has to add /Users/brianp/work to the PYTHONPATH to do that.

This is how pytest deals with the problem of importing test files which are part of a package, and is not really related to pytest.ini or rootdir.

[cepbrian]

ok. I guess my point was just when looking for "how does pytest figure out what to put into python path" the only thing I found in the documentation was "rootdir", which didn't make clear that it WASN'T what was added to python path, nor did anything make clear what WAS.

Like I said initially, this is likely just a documentation issue. There should be an easily found documentation on "what does pytest do with pythonpath?" I assumed that "rootdir" was used for pythonpath, because nothing else seemed to be, and it seemed reasonable... Again, a documentation issue.

[nicoddemus]

I see, the confusion is understandable; like I said this is a common source of confusion among users.

Noted about how to improve that on the docs, that's an area where we definitely need improvement!

I guess there are two points that deserve some better docs:

1. Make it clearer what rootdir is and what it isn't.
2. When does pytest add things to PYTHONPATH, and why.

What do you think?

[cepbrian]

yes! thanks!