Coveragepy: KeyError in qualname_from_frame starting from 5.0a6

[fersarr]

Describe the bug
After upgrading from coveragepy 5.0a5-ish to to 5.0a6, we started having a few failing builds with something that looks as follows:

Traceback (most recent call last):
  File "/eggpath/p/pytest-3.1.0-py2.7.egg/_pytest/config.py", line 367, in _importconftest
    mod = conftestpath.pyimport()
  File "/eggpath/p/py-1.4.31-py2.7.egg/py/_path/local.py", line 650, in pyimport
    __import__(modname)
  File "/eggpath/p/pytest-3.1.0-py2.7.egg/_pytest/assertion/rewrite.py", line 216, in load_module
    py.builtin.exec_(co, mod.__dict__)
  File "/eggpath/p/py-1.4.31-py2.7.egg/py/_builtin.py", line 221, in exec_
    exec2(obj, globals, locals)
  File "<string>", line 7, in exec2
  File "/workspace_path//test-path/tests/conftest.py", line 6, in <module>
    import xxxx
   ...
   ...
  File "/eggpath/t/tables-3.2.2-py2.7-linux-x86_64.egg/tables/__init__.py", line 123, in <module>
    from tables.file import File, open_file, copy_file, openFile, copyFile
  File "/eggpath/t/tables-3.2.2-py2.7-linux-x86_64.egg/tables/file.py", line 31, in <module>
    import numexpr
  File "/eggpath/n/numexpr-2.6.4-py2.7-linux-x86_64.egg/numexpr/__init__.py", line 42, in <module>
    from numexpr.tests import test, print_versions
  File "/eggpath/n/numexpr-2.6.4-py2.7-linux-x86_64.egg/numexpr/tests/__init__.py", line 11, in <module>
    from numexpr.tests.test_numexpr import test, print_versions
  File "/eggpath/p/pytest-3.1.0-py2.7.egg/_pytest/assertion/rewrite.py", line 216, in load_module
    py.builtin.exec_(co, mod.__dict__)
  File "/eggpath/p/py-1.4.31-py2.7.egg/py/_builtin.py", line 221, in exec_
    exec2(obj, globals, locals)
  File "<string>", line 7, in exec2
  File "/eggpath/n/numexpr-2.6.4-py2.7-linux-x86_64.egg/numexpr/tests/test_numexpr.py", line 44, in <module>
    class test_numexpr(TestCase):
  File "/workspace_path//coveragepy/coverage/context.py", line 40, in should_start_context_test_function
    return qualname_from_frame(frame)
  File "/workspace_path//coveragepy/coverage/context.py", line 49, in qualname_from_frame
    func = frame.f_globals[fname]
KeyError: 'test_numexpr'
ERROR: could not load /workspace_path//test-path/tests/conftest.py

In case it helps, both failed builds have these lines the log:

       return qualname_from_frame(frame)
       func = frame.f_globals[fname]

and both also mention KeyError: 'test_numexpr'

I really struggled to understand why this is happening, BUT I went through all commits in coveragepy and, at least, I found that the commit that breaks it is the following:
https://github.com/nedbat/coveragepy/commit/a9f5f7fadacaa8a84b4ac247e79bcb6f29935bb1

The command that was run to cause this is:

coverage run -p --branch  /path.../bin/py.test -svvv tests/unit/

Please let me know if I can provide any other information that might help you

Expected behavior
All tests should be collected and pass, as it happens when using coveragepy from a point in time before the commit mentioned above.

Additional context

• Python: seems to happen on both python versions I tried 3.6.5 and 2.7.13
• pytest 3.1.0

[nedbat]

Any chance I can get the project that failed? So that I can run the tests myself? Is numexpr your project?

[fersarr]

Unfortunately both of the projects that failed are internal to my company. Both use numexpr (https://pypi.org/project/numexpr/). I will try to keep digging to see if I can provide more information.

[nedbat]

I see what's going on here: there's a class named "test_numexpr" (unusual name for a class). It's being called, so coverage thinks it's a function. I'll try making a small reproducer.

[nedbat]

Ah, it's easy to make happen:

$ cat bug829.py
class test_it(object):
    pass

$ cat .coveragerc
[run]
dynamic_context = test_function

$ coverage run bug829.py
Traceback (most recent call last):
  File "bug829.py", line 1, in <module>
    class test_it(object):
  File "/Users/ned/coverage/trunk/coverage/context.py", line 41, in should_start_context_test_function
    return qualname_from_frame(frame)
  File "/Users/ned/coverage/trunk/coverage/context.py", line 50, in qualname_from_frame
    func = frame.f_globals[fname]
KeyError: 'test_it'

Thanks for the interesting report! :)

[fersarr]

Wow! That's good news! I honestly didn't know what else to try to figure this one out! So glad :)

[nedbat]

Fixed in 07c3c5db.

[fersarr]

Just to know, no rush, when do you think there will be a new release containing this fix?

[nedbat]

This is in 5.0a7.