Pytest: unexpected indent of test function with docstring

[515hikaru]

When run test with docstring, FAILURE message indent is unexpected.

For example,

import unittest
class TestSome(unittest.TestCase):

    def test_bar(self):
        """
        This test fails.
        """
        self.assertEqual(1, 3)

run above test with pytest, following message is shown.

    def test_foo(self):
        """
            This test fails.
            """
>       self.assertEqual(1, 3)
E       AssertionError: 1 != 3

But I expected message like below.

    def test_foo(self):
        """
        This test fails.
        """
>       self.assertEqual(1, 3)
E       AssertionError: 1 != 3

I found out the cause is probably tokenize.generate_tokens. (It is used src/_pytest/_code/source.deindent ). However, I do not know how to fix it.

Version

• macOS 10.14
• Python: 3.6.5
• pytest: 3.8.1

[asottile]

I can reproduce this as well.

While debugging we _have_ the right source:

(Pdb) print('\n'.join(self.frame.code.fullsource))
import unittest
class TestSome(unittest.TestCase):

    def test_bar(self):
        """
        This test fails.
        """
        self.assertEqual(1, 3)
(Pdb) print(source[start:end])
    def test_bar(self):
        """
        This test fails.
        """
        self.assertEqual(1, 3)

And here's where it goes awry:

(Pdb) n
> /home/asottile/workspace/pytest/src/_pytest/_code/code.py(540)_getentrysource()
-> source = source.deindent()
(Pdb) n
> /home/asottile/workspace/pytest/src/_pytest/_code/code.py(541)_getentrysource()
-> return source
(Pdb) print(source)
def test_bar(self):
    """
        This test fails.
        """
    self.assertEqual(1, 3)

This does appear intentional, but weird. deindent uses the tokenizer to do indentation changes. Since a docstring is a multi line string, the whitespace _is_ important and so the start of the token becomes deindented but not the whitespace contained.

And actually, here's a great example where the deindent actually introduces a misleading output:

def test_bar():
    x = """\
    sneaky!"""
    assert x == '        sneaky!'

$ pytest t.py 
============================= test session starts ==============================
platform linux -- Python 3.6.5, pytest-3.8.3.dev1+gdf435fa8.d20181002, py-1.6.0, pluggy-0.7.1
rootdir: /tmp/t, inifile:
collected 1 item                                                               

t.py F                                                                   [100%]

=================================== FAILURES ===================================
___________________________________ test_bar ___________________________________

    def test_bar():
        x = """\
        sneaky!"""
>       assert x == '        sneaky!'
E       AssertionError: assert '    sneaky!' == '        sneaky!'
E         -     sneaky!
E         +         sneaky!
E         ? ++++

if I were to copy paste the code out of that:

>>> x = """\
...         sneaky!"""
>>> assert x == '        sneaky!'
>>> 

it passes! wow!

my inkling is that this is one of those "damned if you do damned if you don't" situations I think we should make the indentation make sense with the source text and remove our fancy tokenize dedent logic and just use textwrap.dedent -- @nicoddemus @RonnyPfannschmidt thoughts?

[nicoddemus]

Hehehe I was just debugging this one. 😁

I replaced deindent with this:

def deindent(lines, offset=None):
    import textwrap
    return textwrap.dedent('\n'.join(lines)).split('\n')

With this new implementation I get the expected output:

============================================ FAILURES =============================================
________________________________________ TestSome.test_bar ________________________________________

self = <test-dedent.TestSome instance at 0x04598490>

    def test_bar(self):
        """
        This test fails.
        """
>       assert 1 == 3
E       assert 1 == 3

test-dedent.py:8: AssertionError
==================================== 1 failed in 0.03 seconds =====================================

While the original implementation gives:

============================================ FAILURES =============================================
________________________________________ TestSome.test_bar ________________________________________

self = <test-dedent.TestSome instance at 0x047D5508>

    def test_bar(self):
        """
            This test fails.
            """
>       assert 1 == 3
E       assert 1 == 3

test-dedent.py:8: AssertionError
==================================== 1 failed in 0.03 seconds =====================================

But 4 tests in test_source.py fail because of that. One of the tests actually seem to be incorrect and the new output is the correct one, the others I didn't take a look yet.

Want to assume this one @asottile? I won't have much more time to work on this one anyway.

[nicoddemus]

Btw: my impression is that the deindent function was written way before textwrap.dedent was added to the standard library, so it might be just the case to test with a few new scenarios and update the implementation to use textwrap.dedent if it shows the correct output, and update the existing tests accordingly.

[asottile]

:+1:

[asottile]

4069 attempts a fix at this!

[asottile]

weird, github didn't auto close this despite the "Resolves #xxxx" description. anyway, it is fixed now :tada: thanks again for the issue @515hikaru !

[asottile]

or my browser is confused, disregard