pytest silently chooses the wrong fixture when two plugins declare a fixture with the same name

[burnpanck]

The magic name-matching of pytest's fixtures can lead to namespace clashes, apart from being unpythonic (see #3834). Getting a wrong fixture may lead to hard-to-debug errors, especially if one does not know the plugin code and therefore does not know if one plugin may legitimately call the other plugin's code. Thus, I believe that pytest should not allow a fixture to be used whose name clashes between two declarations (Python Zen: In the face of ambiguity, refuse the temptation to guess.). Instead, it should raise an explicit error, including a hint to the test-writer how he may resolve the ambiguity.

The environment where this bug was observed was the following:

python 3.6.1
This is pytest version 3.5.0
  pytest-sanic-0.1.13
  pytest-benchmark-3.1.1
  pytest-aiohttp-0.3.0

where both aiohttp and sanic provide a test_client fixture.

[Zac-HD]

I support pytest raising an explicit error whenever it would otherwise choose between multiple fixtures 👍

IMO this has never worked and we could go to an error straight away, but if others disagree we might need a deprecation period first.

[RonnyPfannschmidt]

we have to account for layered fixture overrides however

def myfixture(myfixture): ... is supported as a customization system for plugins

additionally i believe pytest-selenium uses those overrides in order to allow local testsuites to customize certain settings

[Zac-HD]

😱... this isn't a fatal problem though, because in this case it's always the more-recently-defined fixture that should be chosen.

We can therefore ignore "parent" fixtures, but still error on unrelated or "sibling" fixtures.

[RonnyPfannschmidt]

detecting sieblings correctly will be tricky

i propose to introduce a flag for fixtures that are supposed to be overridden, and then only warning on fixtures that override fixtures that are not meant for overriding

[RonnyPfannschmidt]

that way we can count the existence of myfixture(myfixture) as externally allowed override and and a fixture(..., allow_override=True) marking as internally allowed override

[Zac-HD]

detecting siblings correctly will be tricky

So long as we can tell what (if any) fixtures a given fixture layers on, it's just a matter of walking back up that tree looking for the conflicting fixtures. And if there are no conflicting fixtures we don't need to do anything, so the performance should be fine.

Creating a distinction between fixtures that can and cannot be overridden seems likely to cause lots of subtle problems about ordering and sequences of fixtures, so I'd avoid it.

[RonnyPfannschmidt]

we already have a situation where overriding is used as a feature, so we cant break it
we have explicit overrides where we take in the original fixture as a input value, and we have implicit overrides, where a different definition is supposed to be allowed to be overridden

from my pov we want to to warn of all overrides but

• we want to warn which fixture was overridden,
• we want to ignore an override if the override takes the original fixture as an argument
• we want to ignore the override if the original fixture declares that it is supposed to be overridden

[Zac-HD]

Yep, that sounds like a good plan :rocket: