pytest.raises - casting to string masks the DID NOT RAISE error

[perrygeo]

Given this test code

import pytest

def test_one():
    with pytest.raises(TypeError) as e:
        assert 'something' in str(e)

I would expect to see Failed: DID NOT RAISE <type 'exceptions.TypeError'>

Instead, using pytest==3.2.2, I get the following failure:

====================================== FAILURES =======================================
______________________________________ test_one _______________________________________

    def test_one():
        with pytest.raises(TypeError) as e:
>           assert 'something' in str(e)

tests/test_temp.py:5:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <ExceptionInfo AttributeError tblen=2>

    def __str__(self):
>       entry = self.traceback[-1]
E       AttributeError: 'ExceptionInfo' object has no attribute 'traceback'

../../env/dev27/lib/python2.7/site-packages/_pytest/_code/code.py:438: AttributeError
============================== 1 failed in 0.04 seconds ===============================

Casting to the str() is the culprit here - without the str function call, the test fails in the expected way. Is there any way to make casting the exception info object not fail in this case?

[The-Compiler]

I don't see how that code could work - you're checking the exception info before any exception happened. However, I agree pytest should show a nicer error here.

[perrygeo]

@The-Compiler the code was just an example to illustrate the case where no exception was raised. A more complete example might be:

def test_one():
    with pytest.raises(TypeError) as e:
        'a string' > 0
        assert 'not supported' in str(e)

which will pass on Python 3.6 but fail with AttributeError: 'ExceptionInfo' object has no attribute 'traceback' on Python 2.7 when it should fail with DID NOT RAISE.

[The-Compiler]

I'll have to disagree. If an exception happens inside a pytest.raises block before it was finished, I expect pytest to show me that exception.

[obestwalter]

I think this is a bit of a tricky corner. My current understanding is that it makes no sense to try to acces the exception info inside the pytest.raises block. Correct? As I understand it there are only two possibilities regarding accessing that object in the block:

1. No exception has been raised (yet), so I have no reason to access the exception info
2. An exception has been raised and the pytest.raises block has been exited due to that.

To clarify what I mean:

def test_one():
    with pytest.raises(TypeError) as e:
        raise TypeError("bla")
        # I can do what I want after an exception is raised
        # because this will never be executed
        assert e.args[0] == 'bla' 
        argwargl schmurp schmorp

This is why I have to do this:

def test_one():
    with pytest.raises(TypeError) as e:
        raise TypeError("bla")
    assert e.args[0] == 'bla'

In the docs it is shown how to do it correctly (access the exception info outside of the pytest.raises block) but it is not explicitly stated why this is necessary: https://docs.pytest.org/en/latest/assert.html#assertions-about-expected-exceptions - so maybe that could be improved a bit there. by pointing that out explicitly, I am not sure though.

[perrygeo]

If an exception happens inside a pytest.raises block before it was finished, I expect pytest to show me that exception.

Fully agree. My point is that str(e) of an ExceptionInfo object should not raise an exception in the first place.

access the exception info outside of the pytest.raises block

This sounds like a good solution - asserting that the exception is raised and asserting something about the exception are kept separated. Thanks @obestwalter.

[The-Compiler]

FWIW I think accessing the ExceptionInfo object inside the block should show a clear error (that is in the cases we can do so, i.e. when no other exception happened before)

[asottile]

this was fixed at some point, here's the new output:

$ pytest !$
pytest t.py
============================= test session starts ==============================
platform linux -- Python 3.8.2, pytest-5.4.3, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/x
collected 1 item                                                               

t.py F                                                                   [100%]

=================================== FAILURES ===================================
___________________________________ test_one ___________________________________

    def test_one():
        with pytest.raises(TypeError) as e:
>           assert 'something' in str(e)
E           AssertionError: assert 'something' in '<ExceptionInfo for raises contextmanager>'
E            +  where '<ExceptionInfo for raises contextmanager>' = str(<ExceptionInfo for raises contextmanager>)

t.py:5: AssertionError
=========================== short test summary info ============================
FAILED t.py::test_one - AssertionError: assert 'something' in '<ExceptionInfo...
============================== 1 failed in 0.13s ===============================