Pytest: Failed to collect tests on Windows (tests are collected OK on Linux or macOS)

[pombredanne]

In the scancode-toolkit, I run a fairly large tests suite on Windows, Linux and macOS (with Travis and Appveyor).
I just noticed that some tests are NOT collected at all on Windows on Appveyor (and I confirmed this on plain Windows VM).
Here are two output of running the same --collect-only command on both OSes:

• py.test -vvs tests/licensedcode/test_detection_datadriven.py --collect-only linux-ldd.txt has 2000+ tests collected
• py.test -vvs --collect-only tests\licensedcode\test_detection_datadriven.py > windows-ldd.txt windows-ldd.txt has none of these

The thing about theses tests is that these are parametric tests (not using py.test features for this) where tests methods are attached to a test class at import time .

There are other places where similar tests are collected alright.

To reproduce this issue :

• ensure that you have a Python 2.7 installation
• clone or download scancode-toolkit
• run ./configure (or configure on Windows): this will create a local virtualenv and install all the bundled deps at once
• on POSIX, run source bin/activate
• from that point on you are in a virtualenv with py.test installed and you can issue the py.test commands above.

The default py.test version used ATM is 3.5.1

I tested the bug also with the latest 3.8.0 on windows and that's the same problem.

[pombredanne]

@nicoddemus any idea of what would be the root cause for this?

[nicoddemus]

Hi @pombredanne,

Sorry for the delay, this has slipped through the cracks it seems.

I've debugged the code and noticed this is not a pytest problem, because I can reproduce the problem without pytest:

from unittest import TestLoader
import test_detection_datadriven

loader = TestLoader()
for x in loader.getTestCaseNames(test_detection_datadriven.TestLicenseDataDriven):
    print(x)

This prints only:

Digging further, I noticed that the tests methods being created by build_tests all had the same name (test_detection). This was caused by a naive str.partition call which used a hard coded unix path ('licensedcode/data/').

I've opened https://github.com/nexB/scancode-toolkit/pull/1241 with a fix. 👍

Closing this now as it is not a pytest issue.

[pombredanne]

@nicoddemus no need to be sorry: I am very grateful for py.test. It positively rocks. Sorry for the churn of having incorrectly qualified this as a py.test bug and thank you for the fix!