Pytest: Fixture tear-down order broken in 5.3.3

[MarcSchmitzer]

A change in pytest 5.3.3 (99180939feb, to be precise) broke fixture teardown for us. What happens is that fixture A depends on fixture B, but B gets torn down before A. This means that B is already torn down when A's tear-down code executes. Both fixtures have the same scope.
Reverting 99180939feb on top of 5.3.3 restores the old behavior.

The test suite in question is quite involved. Sadly I haven't been able to create a minimal reproducer so far.

The contents of the virtualenv are here: pip-list.txt.

Pytest Version is 5.3.3, on Ubuntu 18.04.3, Python 3.6.8

[The-Compiler]

Duplicate of #6492

[nicoddemus]

Thanks @MarcSchmitzer, while it does seem like a duplicate, meanwhile if you can get a minimal reproducer it would be great; #6492 is under investigation still, so a reproducer would help a lot.

[blueyed]

/cc @SalmonMode this also seems not like a real duplicate of #6492. Let's verify/triage this separately for now.

[SalmonMode]

@MarcSchmitzer if possible, are you able to run pytest --setup-plan with pytest 5.3.2 and pytest 5.3.3 so we can see the difference?

[MarcSchmitzer]

Plans are here: plan-5.3.2.txt plan-5.3.3.txt. There's a lot of custom fixtures in there, I'm afraid. The relevant change is the second one: The module_conn_pool_factory fixture depends on the connection_service fixture. In the 5.3.3 plan, the service gets torn down first, though.

Edited to add: The plan produced by 5.3.4 is identical to the 5.3.2 plan, by the way.

[MarcSchmitzer]

I've been trying to pare down the test suite to isolate the problem, and it's really weird. I can make 5.3.3 produce a correct plan by removing test functions even though doing that doesn't change the set of used fixtures. This is so arbitrary, it almost seems like a dictionary order issue. But I'm on Python 3.6 here, so dictionary iteration should be in insertion order, right?

[blueyed]

@MarcSchmitzer

it almost seems like a dictionary order issue.

That matches what I've seen in pytest-django where the fixture order used in args is not deterministic anymore (flaky) it seems.

But I'm on Python 3.6 here, so dictionary iteration should be in insertion order, right?

Yes.

[MarcSchmitzer]

dictionary iteration should be in insertion order

But set iteration still appears to be independent of insertion order (but dependent on the hash seed). So maybe that's it..?

[SalmonMode]

@MarcSchmitzer a very interesting example. Thank you! Are you by any chance doing anything with pytest hooks, request.addfinalizer, request.getfixturevalue, or anything out of the ordinary?

[MarcSchmitzer]

@SalmonMode

Are you by any chance doing anything with pytest hooks, request.addfinalizer, request.getfixturevalue, or anything out of the ordinary?

I have removed the few instances of request.addfinalizer that were left from before we started using yield fixtures, without effect. Apart from that, I'd say we're not doing anything unusual.

[blueyed]

@MarcSchmitzer can you try https://github.com/pytest-dev/pytest/pull/6551, please?

[MarcSchmitzer]

@blueyed

@MarcSchmitzer can you try #6551, please?

Sorry for the late response. I tried it and it does fix the problem. The fixtures are torn down in the correct order and the test suite passes without problems.