Pytest-django: Tests fail since pytest-django 3.4 for django-extensions

[michael-k]

I noticed that the tests of django-extensions@master fail since pytest-django 3.4 was released. See eg. https://travis-ci.org/michael-k/django-extensions/builds/417328508

When pinned to pytest-django<3.4 in tox.ini the tests pass: https://travis-ci.org/michael-k/django-extensions/builds/417337453

So far I could not figure out why. It says that No fixture named 'group' found., but the changelog does not include anything helpful: https://github.com/pytest-dev/pytest-django/releases/tag/3.4.0 I also found no such fixture it the pytest-django 3.3.x codebase.

If anyone has a clue where this might be coming from, please let me know or open a PR against the appropriate project.

[blueyed]

Thanks for the info!

I would suggest first git-bisecting pytest-django, and if it is caused by some specific commit in there that might give a good clue.

[blueyed]

Caused by ee40819.
/cc @voidus

[blueyed]

https://github.com/pytest-dev/pytest-django/pull/638 fixes it.

The problem is that "." gets inserted to sys.path there.

[voidus]

That's interesting. I don't really get why inserting . causes pytest to not find the fixtures though. Does something change the working directory?

[blueyed]

The problem in the tests is this:

> …/Vcs/django-extensions/django_extensions/management/commands/runscript.py(130)

 125             def get_directory_basing_on_policy(script_module):                                                                                                                           
 126                 policy = options['dir_policy'] or getattr(settings, 'RUNSCRIPT_CHDIR_POLICY', DirPolicyChoices.NONE)                                                                     
 127                 if policy == DirPolicyChoices.ROOT:                                                                                                                                      
 128                     return settings.BASE_DIR                                                                                                                                             
 129                 elif policy == DirPolicyChoices.EACH:                                                                                                                                    
 130  ->                 return os.path.dirname(inspect.getfile(script_module))                                                                                                               
 131                 else:                                                                                                                                                                    
 132                     return self.current_directory                                                                                                                                        
(Pdb++) script_module
<module 'tests.testapp.scripts.directory_checker_script' from './tests/testapp/scripts/directory_checker_script.py'>
(Pdb++)  os.path.dirname(inspect.getfile(script_module))
'./tests/testapp/scripts'
(Pdb++) import os
(Pdb++) os.getcwd()
'…/Vcs/django-extensions/tests'
(Pdb++) 

script_module.__file__ is relative, and cwd is tests already (changed in test_each_policy_command_run).

And then there are a bunch of setup ERRORs, because of a missing cd -.

[blueyed]

It seems weird that __file__ is no taking into account the current working directory.
The lookup via "." appears to work, although the directory was changed already, but it is not possible to get to the actual file then?! (without keeping track of changed directory state)

[blueyed]

Releasing 3.4.2 - will take a while due to builds on master being queued first: https://travis-ci.org/pytest-dev/pytest-django/builds/418313779

[blueyed]

@michael-k

It seems like the module is imported before already, and then keeps the relative __file__ when being imported again:

import os
import sys

sys.path.insert(0, '.')

# Importing it before chdir already causes failure.
import imported

os.chdir('/')
import imported
assert imported.__file__ == os.path.abspath(imported.__file__)

[blueyed]

Filed https://bugs.python.org/issue34444 for Python.

[michael-k]

Thanks for looking into this! :+1: