Black: Black removes newline from a multiline docstring.

[omry]

Describe the bug A clear and concise description of what the bug is.

class Foo:
    """foo
    """

Is being reformatted to:

class Foo:
    """foo"""

This is removing a newline.
arguably this is okay because it's a comment, but such comments are (as far as I know) accessible through object inspection.

Expected behavior A clear and concise description of what you expected to happen.
I am expecting black to not remove newlines from strings.

Environment (please complete the following information):

• Version: black==20.8b1
• OS and Python version: Linux/Python 3.8.5

Does this bug also happen on master?
Yes.

[cooperlees]

Hi Omry, thanks for reporting.

Do you explicitly only want this only for docstrings? If so, black respects ‘’.

I.e.

class Foo:
  “””Hello \
  “””

This would not be reformatted.

Black also respects multi lines variables with 20.8b1

foo = “””
Cooper made a string
“””

This is also not reformatted on my tests.

What you’re seeing is expected behavior with the new release I believe.

[omry]

Hi @cooperlees,
This caught me off guard, specifically for my use case it's indeed a comment and don't mind the re-formatting.
I did notice that this works as I expect for variables.

the workaround does not seem to be generating the same string:

s1 = """hello
"""

s2 = """hello\ 
"""
print("s1", repr(s1))
print("s2", repr(s2))

$ python 1.py 
s1 'hello\n'
s2 'hello\\ \n'

[cooperlees]

As usual, you've missed the point of my reply. Docstrings need the '\' to not join on the n, like your original post only showed. Normal multi-lines strings should not be doing what you claim.

I am unable to repro:

If I

echo 's1 = """hello
"""

s2 = """hello\ 
"""
print("s1", repr(s1))
print("s2", repr(s2))
' > /tmp/omry.py

cooper-mbp1:bandersnatch cooper$ /tmp/black/bin/black --diff /tmp/omry.py
All done! ✨ 🍰 ✨
1 file would be left unchanged.

Neither multiline string here are reformatted for me ... Please provide a non docstring repro with 20.8b1 or master please that joins a non docstring mutli-line string and we can try fix it.

[omry]

The problem is specific to docstrings.
I used variables in the repro because it was easier for me to demonstrate that the proposed workaround changes the string like that.
Here is a repro that uses docstrings:

Code:
http://paste.ubuntu.com/p/w6mN4Y2R93/

class Foo:
    """hello
    """

class Bar:
    """hello\
    """

print("Foo" ,repr(Foo.__doc__))
print("Bar", repr(Bar.__doc__))

Output:

$ python 1.py 
Foo 'hello\n    '
Bar 'hello    '
$ black 1.py 
reformatted 1.py
All done! ✨ 🍰 ✨
1 file reformatted.

Trying to repro I possibly hit another bug:
if there is a trailing whitespace after the backslash it causes an internal error.
$ cat 1.py | pastebinit
http://paste.ubuntu.com/p/fW5KjwZrR6/
$ black 1.py
error: cannot format 1.py: INTERNAL ERROR: Black produced invalid code: EOF while scanning triple-quoted string literal (, line 10). Please report a bug on https://github.com/psf/black/issues. This invalid output might be helpful: /tmp/blk_prxe8kni.log
Oh no! 💥 💔 💥
1 file failed to reformat.
```

[ambv]

1. The docstring whitespace change is deliberate.
2. The invalid code produced when there's trailing whitespace after a backslash is indeed a bug in Black we need to fix.

[omry]

Thanks ambv.
I personally prefer the formatted output, but I can imagine some cases where people would care about about the newlines in the docstring.

I think # fmt: off and # fmt: on are sufficient as a workaround in these cases.

[laloch]

Could we please have a switch to disable the docstring reformatting altogether? Removing the leading whitespace may break docstrings containing tabular text. For instance this PLY/Yacc grammar rule:

    def p_flow_stmt(self, p):
        """flow_stmt : break_stmt
                     | continue_stmt
                     | return_stmt
                     | raise_stmt
                     | yield_stmt
        """
        p[0] = p[1]

ends up like:

    def p_flow_stmt(self, p):
        """flow_stmt : break_stmt
        | continue_stmt
        | return_stmt
        | raise_stmt
        | yield_stmt
        """
        p[0] = p[1]

[ambv]

@laloch: ~I cannot reproduce this. Were the grammar rules using tab characters instead of spaces?~

Nevermind, I was on the wrong branch. Yeah, we'll deal with this for the next release.

[ambv]

@laloch In the mean time, a workaround for Black's behavior is starting your docstring text on a newline:

    def p_flow_stmt(self, p):
        """
        flow_stmt : break_stmt
                  | continue_stmt
                  | return_stmt
                  | raise_stmt
                  | yield_stmt
        """
        p[0] = p[1]

[laloch]

Thanks @ambv! We have already resorted to pinning v19.10b0 in our CI setup for now.

[DrGFreeman]

I also noticed that the "corrected" docstring will violate the line-length rule if the the docstring is within three characters of the line-length value. Reproduceable example with line-length = 88:

def f():
    """89 123456789 123456789 123456789 123456789 123456789 123456789 123456789 12345678
    """
    pass

will get changed to

def f():
    """89 123456789 123456789 123456789 123456789 123456789 123456789 123456789 12345678"""
    pass

where the docstring now exceeds 88 characters.