Pytest: "unique basename" error vs __init__.py in test sub-folders is a UX mayhem

[pytestbot]

Originally reported by: Thomas Güttler (BitBucket: thomas-guettler, GitHub: thomas-guettler)

---

Our tests don't have unique basenames, since we use this pattern:

The tests for admin.py is located in tests/unit/test_admin.py

There are several apps in one project which have an admin.py.

This way we get the well known error message:

import file mismatch:
imported module 'test_admin' has this __file__ attribute:
  foo/tests/unit/views/test_admin.py
which is not the same as the test file we want to collect:
 bar/tests/unit/test_admin.py
HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules

Our current solution is to create __init__.py files.

This is against the docs:

avoid “__init__.py” files in your test directories. 
This way your tests can run easily against an installed version of mypkg, 
independently from the installed package if it contains the tests or not.

Please think about this again. For me, the constraint to have unique base names way to hard.

Maybe the best solution is to update the docs:

If you avoid __init__.py files you can ....

I don't think every one should avoid __init__.py files.

Related: http://stackoverflow.com/questions/12582503/py-test-test-discovery-failure-when-tests-in-different-directories-are-called

---

• Bitbucket: https://bitbucket.org/pytest-dev/pytest/issue/529

[pytestbot]

_Original comment by_ Floris Bruynooghe (BitBucket: flub, GitHub: flub):

---

AFIAK your __init__.py files are fine. The docs say you should not have them in your test directory, but if you want subdirectories in your test directory you should make them packages, i.e. have __init__.py in them.

So if the docs don't make this clear enough this sounds like a doc bug to me.

[pytestbot]

_Original comment by_ Thomas Güttler (BitBucket: thomas-guettler, GitHub: thomas-guettler):

---

I thought about this again.

The message _HINT: remove pycache / .pyc files and/or use a unique basename for your test file modules_ should be extended:

#!python

... or add an __init__.py file to the directory.

I guess this would be better, since the error message gets read much more, then the docs.

[pytestbot]

_Original comment by_ Dustin Farris (BitBucket: dustinfarris, GitHub: dustinfarris):

---

Ran into this also. I think both the docs and the error message could be improved since most people are looking for some kind of best practice guidance.

[ssbarnea]

Can someone clarify this. I am totally confused about this. I do know that putting an init.py file inside the tests/ folder is against the documentation but when I run the test I get the weird error message that asks me to put one there or to HINT: remove __pycache__ / .pyc files and/or use a unique basename for your test file modules which is again against python recomandations.

Can we decide on something and document it? The current experience is really not a good one.

I don't have any duplicate test names, I just have a very basic use-case: one single test file with 3 tests inside it, all of them with different names. Sadly, I am forced to use the file inside the test file in order to be able to compose the path towards a test data file that I need for running these tests.

ini_filename = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'sample.ini')

You can see the whole code at https://github.com/ssbarnea/test-configparser -- as this is just a minimal test project which aims to replicate some python bug. Somehow it seems that I managed to also hit a pytest bug in the process 🎱

[nicoddemus]

@ssbarnea, could you post exactly how you are running the tests and the full console output you are getting? I notice you are using Travis and it seems it is working (3 tests,1 failing). I also tried your repository locally and I get the same results.

[ssbarnea]

Here is the trick that triggers this bug: if you run the tests on multiple platforms from the same location you will encounter this. I usually mount my OS X home directory on VMs (Linux and Windows) which allows me to run tests on all platforms without having to copy/push/pull code around.

Some tools do need a bit of turning in order to be able to cope with this surprise. For example you need to remember to create different virtualenvs for each platforms. Like virtualenv env-$(uname -s) but it works. Travis will not be able to spot this because it will run the code on different machines, checking it out each time.

[nicoddemus]

I see, thanks.

Do you still get the message if you manually remove __pycache__and .pyc files as the message suggests?

[scolby33]

I too would appreciate clarification as to why __init__.py would interfere with running tests on an installed version...could someone post a mock console session or something showing this?

I'm having an issue with where to put __init__.py's with my project structured like this:

.
├── setup.py
├── src
│   ├── package_name
│   │   ├── __init__.py
│   │   ├── module_name.py
│   │   ├── etc...
├── tests
│   ├── test_module_name.py
│   ├── constants.py  # contains only global variables assignments
│   ├── fixtures
│   │   ├── A_Fixture.py
│   │   ├── __init__.py  # contains only an __all__ assignment
└── tox.ini

In test_module_name.py I have imports like so:

from tests.constants import *
from tests.fixtures import *

This works under Python3.5, but not under 2.7 (fails with ImportError: No module named tests.constants).

Looking back, I'm not even sure how Python3 manages to find the tests "package" or how to fix this without adding __init__.py to allow for relative imports like from ..constants import *, which isn't allowed in non-package code (causes a ValueError).

UPDATE: Upon further investigation, it appears that I have created a namespace package, which I am still trying to wrap my head around. These were added in Python 3.3, which explains why it doesn't work in 2.7. Further clarification on where it is "safe" to include __init__.py files would still be appreciated.

[RonnyPfannschmidt]

you rely on the python 2.5 implicit namespaces folder with no __init__

that is broken not supported on python2.7

py.test pretends that feature does not exist, and manipulates the import path before test module import

however, since you use a src layout, the __init__.py is not going to hurt you much, since it wont put your checkout into sys.path like it would with a traditional "messy" layout

basically, since you use the src folder, you are saved from the ill effect of _-init__ in a toplevel tests folder

[scolby33]

Just to clarify, do you mean python 2.5 implicit namespaces folder or 3.5 implicit namespaces? Can you point me to more information on this being broken? I'd like to read up on it.

[RonnyPfannschmidt]

whops, i meant python3 implicit namespaces
python2 has no implicit namespaces, one needs a __init__.py with a namespace declaration

unfortunately i don't have links ready with me

[scolby33]

Ah, I understand what you mean now. Thanks for the response. I'll add init's, then.

[juanifioren]

I fix this by removing __init__.py inside my tests folder.