pytest --pdb runs pdb for unittest.expectedFailure tests

[anntzer]

pytest correctly handles unittest's expectedFailure (as an xfail), but pytest --pdb still runs pdb when encountering such a test.

$ cat test_foo.py
from unittest import TestCase, expectedFailure

class TestFoo(TestCase):
    @expectedFailure
    def test_xfail(self):
        self.assertEqual(0, 1)

$ pytest
====================================== test session starts ======================================
platform linux -- Python 3.8.1, pytest-5.3.2, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/foo
collected 1 item                                                                                

test_foo.py x                                                                             [100%]

====================================== 1 xfailed in 0.05s =======================================

$ pytest --pdb
====================================== test session starts ======================================
platform linux -- Python 3.8.1, pytest-5.3.2, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/foo
collected 1 item                                                                                

test_foo.py F
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

self = <test_foo.TestFoo testMethod=test_xfail>

    @expectedFailure
    def test_xfail(self):
>       self.assertEqual(0, 1)
E       AssertionError: 0 != 1

test_foo.py:6: AssertionError
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB post_mortem (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>
> /usr/lib/python3.8/unittest/case.py(905)_baseAssertEqual()
-> raise self.failureException(msg)
(Pdb) 

$ pip list
Package        Version
-------------- -------
attrs          19.3.0 
more-itertools 8.0.2  
packaging      19.2   
pip            19.2.3 
pluggy         0.13.1 
py             1.8.1  
pyparsing      2.4.6  
pytest         5.3.2  
setuptools     41.2.0 
six            1.13.0 
wcwidth        0.1.8

• [X] a detailed description of the bug or suggestion
• [X] output of pip list from the virtual environment you are using
• [X] pytest and operating system versions
• [X] minimal example if possible

[hp310780]

Hello, I had a go at looking at this.

When pytest runs with the --pdb flag, it runs the test case using the debug function (see https://github.com/pytest-dev/pytest/blob/master/src/_pytest/unittest.py#L212). The debug function does not collect errors and handle them in a TestResult object (like pytest without pdb does), it merely passes them through (as expected for a debugging function I guess?), which results in all exceptions being processed and invoking pdb (see https://github.com/pytest-dev/pytest/blob/master/src/_pytest/runner.py#L131).

If anyone has any thoughts on the right way to move forwards with this it'd be appreciated since this is probably expected behaviour for a debugging function?

Please let me know if I've made any missteps here. Thanks!

[blueyed]

Please try the features branch then, where this is different: https://github.com/pytest-dev/pytest/blob/0375c1b728ac8808b424b59e62de5543bbf0994e/src/_pytest/unittest.py#L204-L226

I'm assuming this is fixed there, please report back.

[hp310780]

Yes this issue appears fixed in the features branch, I ran @anntzer 's original minimal example.

[anntzer]

Thanks!