Black: Crash on docstring with tabs

[foozzi]

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

1. Take this file (in text)
2. Run _Black_ on it without any arguments
3. See error

Expected behavior Work

Environment (please complete the following information):

error: cannot format /home/user/cilantropy/dist_worker.py: INTERNAL ERROR: Black produced code that is not equivalent to the source.  Please report a bug on https://github.com/psf/black/issues.  This diff might be helpful: /tmp/blk_rifw8eif.log
Oh no! 💥 💔 💥
10 files left unchanged, 1 file failed to reformat.

• Version: 19.10b0
• OS and Python version: Linux (arch linux)/Python 3.8.3

Does this bug also happen on master? To answer this, you have two options:
pip install git+git://github.com/psf/black

Additional context:
Log:

--- src
+++ dst
@@ -62,11 +62,11 @@
       body=
         Expr(
           value=
             Constant(
               value=
-                'main worker for run subproccess and return result or error\n\n:param cmd: full command for run\t\n:return: result string and status error',  # str
+                'main worker for run subproccess and return result or error\n\n:param cmd: full command for run\n:return: result string and status error',  # str
             )  # /Constant
         )  # /Expr
         Try(
           body=
             Assign(

and my code:

def main_worker(cmd):
    """ 
    main worker for run subproccess and return result or error

    :param cmd: full command for run    
    :return: result string and status error
    """ 

    try:
        cmd_response = subprocess.run(
            cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True
        )
    except subprocess.CalledProcessError as e:
        return "The pip command did not succeed: {stderr}".format(
                stderr=e.stderr.decode("utf-8")
            ), True

    return cmd_response.stdout.decode("utf-8").replace(" "*6,"\n"), False

[hugovk]

This is caused by the tab at the end of a line in the docstring:

---

Here's a minimal example (initial tabs replaced with spaces):

def thing():
    """ 
    line ending in a tab    
    line ending with no whitespace
    """

---

Error does not occur with a tab at the end of the docstring:

def thing():
    """ 
    line ending in a tab    
    """

And does not occur with a space at the end:

def thing():
    """ 
    line ending in a space 
    line ending with no whitespace
    """

[foozzi]

@hugovk thanks!

[hugovk]

You're welcome!

This is still a bug in Black, so we can keep it open to be investigated :)

[hugovk]

git bisect points to a4c11a75e12300abfbe4c36854e450d42bdd1ee7 as the first bad commit, from "Re-indent the contents of docstrings" (#1053).

[JelleZijlstra]

We can probably fix this by loosening the constraints on docstrings when we do the AST comparison, similar to what I did in #1593.

[peterjc]

Updated (again), sorry for the noise.

Confirming a duplicate like #1638 (which may be a different but related issue in black 20.8b1)

[aureliojargas]

Note that the bug also occurs when there's a literal tab in the middle of a line. It seems black converts that to spaces and errors out.

def foo():
    """
    a   b  # it's a literal tab between a and b
    """
    pass

$ black foo.py
error: cannot format foo.py: INTERNAL ERROR: Black produced code that is not equivalent to the source.  Please report a bug on https://github.com/psf/black/issues.  This diff might be helpful: /tmp/blk_q5uiqxs8.log
Oh no! 💥 💔 💥
1 file failed to reformat.

The contents of /tmp/blk_q5uiqxs8.log

--- src
+++ dst
@@ -23,11 +23,11 @@
       body=
         Expr(
           value=
             Constant(
               value=
-                "a\tb  # it's a literal tab between a and b",  # str
+                "a   b  # it's a literal tab between a and b",  # str
             )  # /Constant
         )  # /Expr
         Pass(
         )  # /Pass
       decorator_list=

[manics]

Note also the bug only occurs with a multiline docstring:
This fails:

def f(p):
    """p:   Parameter
    """
    pass

This passes:

def f(p):
    """p:   Parameter"""
    pass

[jparise]

This appears to be caused by the intentional decision (introduced in #1053) to expand tabs to space (using `tabsize=8``, which is the expandtabs() default):

https://github.com/psf/black/blob/6dddbd72414061cde9dd8ee72eac373b7fcf8b54/src/black/__init__.py#L6745-L6747