pytest.raises match produces different result than ex.value when lists are involved

[d2gex]

Operating System: macOS Sierra Version 10.12
Pytest Version: "version": "==3.6.3"
Python Version: 3.6.0

Hi guys,

Not sure if I missed something so apologies in advance if I did. I am testing a routine of mine by checking that a certain exception is raised when a certain scenario occurs. For that I used the context manager _with pytest.raises..._ and the problems occurs when I try to check if the exception message is the one expected.

"_match_" parameter keeps telling me that the pattern has not been found while if I the string comparison is done after context manager using "_str(ex.value)_" the assertion is correct. The problems occurs when I format a message with text and a list - I have not tried other data structures. If I tried only either text or lists, the issue does not appear.

I have read the documentation (https://docs.pytest.org/en/latest/reference.html#pytest-raises) and used match many times before however the formatting of the message was always composed of text and a couple of string types passed with " bla bla bla ".format(string_1, string_2). So the problem seems to be happening when I use "bla bla bla".format(list_1, list_2).

import pytest
import errors

expected_message = "bla {}".format([1, 2, 3])

def foo():
    raise errors.ProtocolDirectiveExchangeError("bla {}".format([1, 2, 3]))


def test_foo_1():
    with pytest.raises(errors.ProtocolDirectiveExchangeError, match=expected_message):
        foo()


def test_foo_2():
    with pytest.raises(errors.ProtocolDirectiveExchangeError) as ex:
        foo()
    assert expected_message == str(ex.value)


platform darwin -- Python 3.6.0, pytest-3.6.0, py-1.5.3, pluggy-0.6.0 -- 
cachedir: .pytest_cache
rootdir: ------
collected 2 items                                                                                                                                                                                                                                                    

test_run.py::test_foo_1 FAILED
test_run.py::test_foo_2 PASSED

============================================================================================================================== FAILURES ==============================================================================================================================
_____________________________________________________________________________________________________________________________ test_foo_1 _____________________________________________________________________________________________________________________________

    def test_foo_1():

        with pytest.raises(errors.ProtocolDirectiveExchangeError, match="bla {}".format([1, 2, 3])):
>           foo()

test_run.py:12: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

    def foo():
>       raise errors.ProtocolDirectiveExchangeError("bla {}".format([1, 2, 3]))
E       errors.ProtocolDirectiveExchangeError: bla [1, 2, 3]

test_run.py:6: ProtocolDirectiveExchangeError

During handling of the above exception, another exception occurred:

    def test_foo_1():

        with pytest.raises(errors.ProtocolDirectiveExchangeError, match="bla {}".format([1, 2, 3])):
>           foo()
E           AssertionError: Pattern 'bla [1, 2, 3]' not found in 'bla [1, 2, 3]'

test_run.py:12: AssertionError

[RonnyPfannschmidt]

match is a regex - please use regex escaping - aka \[\] instead of []

[asottile]

You may find re.escape useful!

[d2gex]

Hi guys,

Thanks a lot. I think I was a sort of misled by the documentation https://docs.pytest.org/en/latest/reference.html#pytest-raises

match – if specified, asserts that the exception matches a _text_ or regex

However when I dug deeper and explored the doc about the ExceptionInfo object returned by the helper, the match documentation on https://docs.pytest.org/en/latest/reference.html#_pytest._code.ExceptionInfo is more explicit:

Match the regular expression ‘regexp’ on the string representation of the exception.

Finally the source code is way more clarifying https://docs.pytest.org/en/latest/_modules/_pytest/_code/code.html#ExceptionInfo.match.

As asottile suggested the following code worked for test_foo_1, but obviously not for test_foo_2.
expected_message = re.escape("bla {}".format([1, 2, 3]))

Lastly once you know the facts, the error message itself is clear enough with "Pattern not found in...".

[RonnyPfannschmidt]

then we should change the docs - thanks for going all the way and gathering the indications and evidence