Pymc3: Failing tests with generators pass

[jonathanhfriedman]

Tests that incorporate generators and inherit from SeededTest can pass even if they should fail. This issue occurs for me both locally and on Travis. For example, this test should clearly never pass:

from .helpers import SeededTest

class TestClassThatShouldFail(SeededTest):
    def test_that_should_fail(self):
        assert 0
        for i in [1/0]:
            assert 0
            yield self.failing_check
        assert 0

    def failing_check(self):
        assert 0

However, it does pass on both Python 2 and Python 3. This seems to affect the tests in, for example, test_distributions.py, where I can't get a test that includes a yield to fail.

Maybe it has something to do with method generators not being supported for subclasses of unittest.TestCase? Either not subclassing from SeededTest or removing the yield both cause the test to fail as expected.

[twiecki]

Any idea @ColCarroll?

[ColCarroll]

ah, you're exactly right @jonathanhfriedman -- see test_step.py, where TestStepMethods is a subclass of object, instead of TestCase. Haven't looked for all examples, but seems like we can either change the superclass to object (and make sure no convenience methods are used), or use test functions. If we were ambitious we might refactor SeededTest to subclass object...

I can help with this, probably tomorrow!

[jonathanhfriedman]

@ColCarroll I would also be happy to take a stab at this if you haven't already started.

[ColCarroll]

Feel free if you want to -- I'll tackle it in ~8hrs if I don't hear anything!

[jonathanhfriedman]

I probably won't be able to work on it by then, so all yours!

[ColCarroll]

1771 thanks for catching -- fixing and actually running the tests increased coverage by 1%.