Black: Black should have an opinion about doc strings

[Lukas0907]

Operating system: Ubuntu 16.04
Python version: 3.6.1
Black version: master
Does also happen on master: yes

Hi,

currently Black doesn't seem to have an opinion about doc strings or rather where the quotes should go.

Black claims for this file for example that it is already formatted:

def test():
    """This is one possibility.

    Very important stuff here.
    """


def test2():
    """This is another one possibility.

    Very important stuff here."""


def test3():
    """
    This is another one possibility.

    Very important stuff here.
    """


def test4():
    """
    What about this?
    """


def test5():
    """What about this?

    Some people also like to have an empty line at the end of the doc string.

    """

The tool pydocstyle (in the spirit of PEP-0257) at least complains that the closing quotes should go on a separate line and if the docstring fits one line it should only span one line.

It would be nice if that could be incorporated in Black as well.

Thanks for the great work!

Lukas.

[rinslow]

I will suggest my opinion:

"""One-liner.

Continuation for the one-liner. (Optional)

Title:
|4 SPACES|one line. might be really really really really really really really really
|8 SPACES| really long
|4 SPACES| next line (something): something something

Note:
|4 SPACES| this is just my opinion but obviously it's the best one
"""

And also:
"""One-liner documentation is closed on the same-line."""

[ambv]

So I came here to complain that docstrings are out of scope but reading your comment actually makes me think what you're asking for is reasonable :smile:

Specifically:

• if it fits one line, it should;
• if it doesn't, the closing quotes should be on their own line.

I'll think about possible negative effects some more before deciding whether to do this for sure but it looks like some opinions here might be harmless.

@carljm, what do you think?

[carljm]

I think it's reasonable for Black to be opinionated about this, and I think the two rules you named are the right ones to enforce. I would extend the second rule to also have an opinion about opening quotes; I slightly prefer both opening and closing quotes on their own line for a multi-line docstring, but wouldn't object to either variant.

I'm not sure why, but I feel like consistent docstring formatting actually has a huge impact on whether code feels consistently formatted. I guess because they tend to be visibly syntax-highlighted large blocks of text.

[natecox]

I tend to favor single line where applicable:

def somefunc():
    """This is my docstring"""

Multi-line I'd personally prefer the first line be on a new line (i.e., test3 above) but I wouldn't complain about the test format above either.

[skapil]

@ambv Not sure, if anyone is looking into it. Can I give a try to this?

[ambv]

Of course, go for it!

[willingc]

Agree with the approach @ambv and @carljm put forth. Like Carl, I like the longer docstrings to have """ on their own lines. I often put a blank line before the closing """ as it adds a bit of whitespace but it's totally fine not to have the blank line.

[coxley]

+1 to prefer single line if possible, otherwise both opening and closing quotes on their own lines. :)

[sersorrel]

Just to throw it out there, I personally prefer PEP 257 style, and I think that's the right thing to do if Black is aiming to follow PEP 8:

"""if it all fits on one line"""

"""if not: some title

more text
"""

but I've seen a wide variety of other styles in the wild, including these two which I don't think have been mentioned:

""" spaces either side """

"""two line one-liner
"""

I do kind of like having a blank line before the closing quotes, as it makes wrapping the body of the docstring much easier in Vim, though it's probably better to write a Vim plugin that fixes this in Vim rather than influencing Black's style.

[carljm]

@anowlcalledjosh PEP 257 is explicitly agnostic about the placement of opening quotes in a multi-line docstring. To quote: "The summary line may be on the same line as the opening quotes or on the next line."

So Black can choose either of these and be equally PEP 257 compliant:

"""
Summary line.

More lines.
"""

or

"""Summary line.

More lines.
"""

My (weak) preference for the former is based on allowing three more characters in the summary line before hitting a line-length limit.

Interestingly, on the question of blank line before closing quotes, PEP257 requires it on class docstrings but not function docstrings ("Insert a blank line after all docstrings (one-line or multi-line) that document a class") because the docstring should be spaced from the first method. Black also adds a blank line (outside the docstring) before the initial method, though, which PEP257 probably doesn't anticipate. (EDIT on second read, it's not entirely clear if the PEP means for this blank line to be inside or outside the quotes; if outside, then that matches what Black already will do.)

[ambv]

it's not entirely clear if the PEP means for this blank line to be inside or outside the quotes; if outside, then that matches what Black already will do.

From my read I thought I'm doing what the PEP is suggesting.

[natecox]

I'm moderately certain PEP is telling us to add a blank line after the closing quotes

[lithammer]

Worth thinking about as well is that the docstring format will have impact on __doc__. So for example this docstring:

"""
Summary line.

More lines.
"""

Will produce this __doc__:

'\n    Summary line\n\n    More lines.\n    '

While if you start the summary line immediately you don't get the \n at the start of the __doc__ attribute.

[ambv]

@renstrom, that used to bother me in the past when help() used to write out the contents of docstrings without calling lstrip() on it first. Now it's less of an issue and tools like PyCharm already correctly strip left whitespace, too.

[brianv0]

Support was recently merged into pycodestyle to support a doc string line length which differs from that of the code line length in PyCQA/pycodestyle#674. It'd be nice to have some tooling and an opinion around that.

The primary use case is to comply with a 72 character docstring limit in pep8, but it's also immensely useful if you allow line lengths of 120 but just want to make sure your docstrings are below 80 so it plays nicely with pydoc.

[sginn]

In response to https://github.com/ambv/black/issues/144#issuecomment-387883874, The blank line should be removed if it's a docstring for a function, but retained for a class.

After running black on our codebase, Flake8 is complaining about D202 No blank lines allowed after function docstring.

def req_required(method=None):
    """Configure reqparse for the request."""

    def decorator(f):
        pass

Black is inserting that blank line between the docstring first line, but shouldn't, because req_required is not a class.

from PEP-257

Insert a blank line after all docstrings (one-line or multi-line) that document a class -- generally speaking, the class's methods are separated from each other by a single blank line, and the docstring needs to be offset from the first method by a blank line.

One-line Docstrings .... Notes: ... - There's no blank line either before or after the docstring.

[ambv]

Black differentiates inner functions from the rest of the body. If you had other instructions there. It wouldn't put empty lines.

[dirn]

This could be tricky when writing a decorator.

def decorator(f):
    """A decorator."""
    @wraps(f)
    def inner():
        return f()
    return inner

Black will add a line after the docstring, causing pydocstyle to report a D202. It sounds like the way to prevent this is to put something else between the two defs, but there isn't always anything to put there as the outer function often just returns the inner one. (I've tried both with and without wraps and the result is the same.)

[auscompgeek]

For those waiting on this, docformatter might interest you: https://github.com/myint/docformatter

[TiemenSch]

Regardless of which layout you choose, I think too many debatable cases will arise if Black would try to enforce a line-length in your docstrings' bodies, for example. Some examples concerning the body text:

• If you would like to be sticking to 80, but your one-liner is too long. Should it change something? Black isn't a linter, but there is no clear solution either.
• Line length in subsequent blocks (Arguments and what not). It is more common to continue lines there with an extra indent. But not every continuation needs to indent further and further. Say a sentence spanning three lines, then the second line needs to get an (additional) indent, but the third shouldn't indent any further. It's hard to detect when a sentence/remark ends (unless you enforce a "." at the end of every sentence)
• Trailing whitespace? --> I feel this can be trimmed without doing to much harm?

Gets messy real quick :)

Perhaps there are too many docstring styles to enforce/output something more useful than the input in the bodies. Who knows whether someone is using Epydoc, Sphinx, Google or any other style.

So I think it's best for the "opinion" of Black to stick to the bare layout, like it is above (with an option to turn doc formatting off). And trailing whitespace could be trimmed without causing trouble AFAIK.

My preference would be one-liners and test() for the multi-line one.

[bancek]

I've implemented docstring indent fixing in my fork.

https://github.com/bancek/black/commit/0889797612d14d9dc5954df6a7de73076c7cb353

[maxalbert]

Agreed it would be great if black could enforce consistency of spacing around opening/closing quotes (I also agree with @TiemenSch that it's likely not a good idea to touch the contents of the docstrings themselves).

Just to note that imho readability should also be a consideration (if not the primary one?).

Personally I find docstrings the most readable if opening and closing quotes are on separate lines (as in test3) - even for single-line docstrings - because your eyes can immediately start parsing the text, rather than having to search for the end of the opening quotes first. But others' opinions may of course differ (and not sure whether there are any other considerations around editors and automated tools).

[ndevenish]

I've implemented docstring indent fixing in my fork.

bancek@0889797

Thanks for posting this, we just converted a large project from two-space indentation directly to black, and this stopped all the docstrings looking weird.

[dimaqq]

Cheers, @bancek this is exactly what annoyed me and why I came to report a bug :)

TL;DR it fixes the space in:

def foo():
    """First line.
[space]
    Second line.
    """

bancek's branch doesn't take opinion on where the opening/closing tripe-double-quote should be, or how long lines in the docstring could be.

[dougthor42]

One thing that should be noted: the automatic formatting of docstrings needs to not break document converters (such as Sphinx) and their expected formats.

[TiemenSch]

Since Black promises to be uncompromising, I think the best bet for this feature is to have a docstring formatter package that is 'Black compatible' so to say. Black could probably expose some plugin functionality, but that would just be a stub then anyways?

[ambv]

Reformatting strings will be handled as part of #182, as solved by #1132. Blank line handling will be handled as part of #450.

[Peque]

@ambv Could you point out where the docstring issue is being or has been addressed? It seems those issues are about long strings and blank lines, not about docstring formatting.

[ambv]

You're right that what #182 describes is not enough.

[ambv]

Interesting bit from #745: format doctests in docstrings, too. It is an open question whether we can approach this and if yes, how. The rules for console-based Python are different. For example, we wouldn't format string reprs which use single quotes; there's >>> and ... prompts, and so on. But it's an interesting thing to decide.

@mattharrison even has a script that already uses Black for this purpose: https://gist.github.com/mattharrison/2a1a263597d80e99cf85e898b800ec32

[Mattwmaster58]

I skimmed this whole issue and didn't seem to see any definitve decision on formatting docstrings. Has one been made? If not, could some sort of poll be made to check the general black-formatter population?

[max-sixty]

FYI this library from @keewis will format ~docstrings~ doctests in docstrings using black https://github.com/keewis/black-doctest

[Peque]

@max-sixty That seems to format code in doctest blocks only.

[keewis]

for a older, a little bit more mature tool, see https://github.com/asottile/blacken-docs. Instead of grouping lines and formatting expressions one at a time it uses regular expressions to parse and format continuous blocks of code. That means it might be faster, but also that adding doctest support is tricky.

I only found it after I had written my own tool which currently specializes in formatting doctest blocks (I might add support for other formats if I need them or if someone puts in a PR).

Note that neither will reformat docstrings, only code blocks in docstrings or documentation files.

Edit: black-doctest it has been moved to blackdoc

[laike9m]

Since long string support has been added, I'm looking forward to this one. Docstring support is the most needed feature for me.

[rpkilby]

I would extend the second rule to also have an opinion about opening quotes; I slightly prefer both opening and closing quotes on their own line for a multi-line docstring, but wouldn't object to either variant.

Google's docstring style places the subject on the same line as the opening quotes. Enforcing that the subject be on a separate line would make black incompatible with that docstring style.

[dougthor42]

Enforcing that the subject be on a separate line would make black incompatible with [Google's] docstring style.

Is black already compatible with Google's general style guide? If not, then we shouldn't worry about that.

It seems like the general consensus is that opening and closing quotes should be on their own line for multi-line docstrings.

[lithammer]

It seems like the general consensus is that opening and closing quotes should be on their own line for multi-line docstrings.

I beg to differ. Even CPython itself is using this form (which I also prefer):

"""Lorem ipsum dolor sit amet."""

```python
"""Lorem ipsum dolor sit amet.

Dignissim diam quis enim lobortis scelerisque fermentum dui. Eu turpis egestas
pretium aenean pharetra. Dictum non consectetur a erat nam at lectus.
"""

[Peque]

@lithammer CPython's docstrings are so ugly though... :stuck_out_tongue_winking_eye:

PEP-257 says both are correct for multi-line docstrings:

The summary line may be on the same line as the opening quotes or on the next line

See also how PEP-287 uses a different line for multi-line docstrings.

I personally prefer Google's style, but, even better, NumPy's. So I think this should be open for discussion.

Also, Black has already previously ignored "official" recommendations (like sticking to 79 characters), so who knows what this could end up being! :joy:

[bfelbo]

Have you considered the usage of docstrings when writing SQL queries and other long strings inside the code?

We use docstrings extensively for this purpose and would really prefer to have opening and closing quotes on their own line for multi-line docstrings. E.g.

"""
SELECT column1, column2
FROM table_name
WHERE cond1;
"""

is much preferred over

"""SELECT column1, column2
FROM table_name
WHERE cond1;
"""

[Mattwmaster58]

Maybe there should be some sort of poll for this as many people are split? Thumbs up on this comment for quotes + subject on same line, ie:

"""subject
body
"""

And thumbs down for quotes + subject separated, ie:

"""
subject
body
"""

[Eric-Arellano]

With multiline strings that aren't docstrings, would \ make any difference in formatting? I usually use the style:

"""\
SELECT column1, column2
FROM table_name
WHERE cond1;
"""

If this is true, then you would have a mechanism for people to still have separated lines with multiline strings, but to still use the """Lorem ipsum dolor sit amet.""" style for docstring.

[rpkilby]

Per https://github.com/psf/black/issues/144#issuecomment-382552074, the second point was suggesting to enforce that closing quotes exist on their own line. Carl had separately suggested that opening quotes also be on their own line, and my point was that this would lock off at least one docstring style.

I don't think the general consensus has been to choose one or the other - I was just offering an argument against his stated minor preference. Ideally (in my mind), both would be viable styles. At best, black should enforce consistency but leave the heavy lifting to a docstring checker.

[sersorrel]

It's also probably worth thinking about whether this is solely about _docstrings_ (as in, a string literal at the start of a function, whether triple-quoted or not), or about triple-quoted strings more generally – @bfelbo it seems unlikely that you're writing SQL queries in docstrings, for example.

```python
def foo():
"""This is a docstring."""
return textwrap.dedent("""This,
however,
isn't!""")

[ndevenish]

At best, black should enforce consistency but leave the heavy lifting to a docstring checker.

Right, I guess it depends _how opinionated_ you want black to be over this issue? I'd expect it to handle things like

• docstring indentation (problem when reformatting two-space code in particular)
• Triple quotes
• Trailing spaces within docstring (general strings feels unsafe)
• Multiple blank lines at beginning?
• Position of trailing quote?
• Maybe single line docstring format..

but anything more, and especially inner-structure, and especially wrapping feels a bit like overreach, and without a single decision-maker massively liable to descend into bikeshedding...

Excepting single-line docstrings, I tend to feel that inline/next-line looks good/bad depending on length and form of contents... maybe that is an argument _for_ black to take that decision away.

[bfelbo]

It's also probably worth thinking about whether this is solely about _docstrings_ (as in, a string literal at the start of a function, whether triple-quoted or not), or about triple-quoted strings more generally – @bfelbo it seems unlikely that you're writing SQL queries in docstrings, for example.

def foo():
    """This is a docstring."""
    return textwrap.dedent("""This,
                              however,
                              isn't!""")

True. We're writing them as triple-quoted strings, not docstrings. Nevertheless, it would be nice to have all triple-quoted strings follow the same style.

[rpkilby]

Right, I guess it depends how opinionated you want black to be over this issue?

Probably not much. Sphinx autodoc integration is powerful, and if black enforces a style that breaks your adopted docstring style, then this would require manually reformatting all your docstrings into a format compatible with these restrictions.

Nevertheless, it would be nice to have all triple-quoted strings follow the same style.

I would disagree with this. Docstrings and block strings serve two different purposes, and the former tend to have an explicit structure. e.g., with Google's docstring style, the first line is the summary, and the rest of the string the extended description.

"""Summary text.

Much longer description with arguments  and examples etc..
"""

In comparison, block strings can contain arbitrary content.

[dougthor42]

[black modifying] anything more, and especially inner-structure, and especially wrapping feels a bit like overreach

if black enforces a style that breaks your adopted docstring style, then this would require manually reformatting all your docstrings into a format compatible with these restrictions.

100% agree. There are too many docstring parsers† for black to try to format docstring contents. The scope of what black does should be limited to things that do not break any of those parsers.

From what I can tell, the Sphinx parser for both NumPyDoc and Google styles (napoleon) accepts either form: "(summary on same line as opening quotes" / "summary on next line".

Thus I think it's safe to for black have an opinion on the "summary on same line" / "summary on next newline" debate. What that opinion is... well that's why we're talking :-)

† I intentionally did not say docstring styles. Neither NumPyDoc nor Google Style explicitly states that the summary must be on the same line as the opening quote††. Instead, they simply imply it via the examples. The examples in Google Style all have the summary on the same line as the quotes. The examples in NumPyDoc have instances of both.

†† If I'm wrong, please show me where it's stated - I couldn't find anything.

---

I tend to feel that inline/next-line looks good/bad depending on length and form of contents... maybe that is an argument for black to take that decision away.

Yes, black should definitely take away the choice.

---

We're writing [SQL queries] as triple-quoted strings, not docstrings. Nevertheless, it would be nice to have all triple-quoted strings follow the same style.

While I understand the wish, I think it's too dangerous to actually implement - formatting something other than a docstring can lead to broken programs.

[bfelbo]

We're writing [SQL queries] as triple-quoted strings, not docstrings. Nevertheless, it would be nice to have all triple-quoted strings follow the same style.

While I understand the wish, I think it's too dangerous to actually implement - formatting something other than a docstring can lead to broken programs.

Agreed. It definitely makes sense for black to only format docstrings.

It would, however, still be great for black to format docstrings in the same way that many developers format their other triple-quoted strings for e.g. SQL queries. At least based on my personal experience, such triple-quoted strings usually have opening and closing quotes on their own line.

[dimaqq]

https://www.python.org/dev/peps/pep-0257/ explicitly talks about docstring processing:

Once trimmed, these docstrings are equivalent:

def foo():
    """A multi-line
    docstring.
    """

def bar():
    """
    A multi-line
    docstring.
    """

Further thoughts: black is opinionated, there should not be any flags or arguments.

On the same note, I don't think black needs to align itself to any other style, whether opinionated (which to choose?) or parametrised (what flags?).

@rpkilby re: Google's style: who uses it? Does your Google internal codebase still use 2-space indentation instead of 4? Is the public style only for open-source projects?

My 2c: let @ambv decide and let's go boldly on!

[danrneal]

Here is a list of things that I would love if black could do related to docstrings. It would also be in compliance with google, numpy and pep257.

1. If a docstring can fit on one line, fit it on one line
2. Remove any blank lines before or after a function docstring
3. Add or remove blank lines so that there is a single blank line between the summary line and description
4. Trade tabs for spaces and make sure entire docstring is properly indented
5. Move multi line docstring closing quotes to their own line
6. Remove trailing whitespace
7. Remove blank lines before a class docstring
8. Indent section headings the same amount as the docstring quotes
9. Remove any blank lines after opening quotes
10. Replace ''' with """
    

    11. Prepend r if any backslashes are in the docstring

(Content-related, out-of scope)

1. Prepend u for Unicode docstring
2. Capitalize first word of the docstring
3. Capitalize section names
4. Add one blank line before and after sections
5. Remove blank lines between a section header and it's content
6. Add a period to the summary line if it doesn't end in .!?

[merwok]

12: all strings are unicode strings now :slightly_smiling_face:

13 to 17: I think analysing docstring contents goes too far.

[danrneal]

That is a good point.... black doesn't change function names if they are out-of-style. I was just thinking of all the times I forgot a period and had my linter yell and me and was thinking that it would be lovely if black added it for me :) But 1-11 should be a good list of things for black to do.

[isaac-friedman]

I was looking for ways to contribute and skimmed this issue. Can I look into at least some of the things on @danrneal 's list?

[ichard26]

@isaac-friedman Go for it! Feel free to tinker with _Black_, maybe it will turn into a PR that might be merged.¹

¹ Please note that not everything on that list has been agreed upon by a core maintainer. So a PR that contains 'non-agreed-upon' changes will probably turn into a design discussion.

[isaac-friedman]

@ichard26 Seems like 1,2 & 5 on the list are the original scope of the issue so I'll start poking at those.

[dougthor42]

@isaac-friedman While not in the original scope, item 10 (use """ instead of ''') is probably also a safe thing to add.

[louwers]

OK, next topic. Is should Black have an opinion on parameter documentation?

E.g.

def func(param1, param2):
    """
    This is a reST style.

    :param param1: this is a first param
    :param param2: this is a second param
    :returns: this is a description of what is returned
    :raises keyError: raises an exception
    """
    pass

More styles here: https://stackoverflow.com/a/24385103

[ssbarnea]

I personally found reST much harder to deal with than MD and with the introduction of type hints the need to follow its param docs seems to create undesired repetition. Sphinx itself seems to now have support for https://www.sphinx-doc.org/en/master/usage/markdown.html but I am not sure if it can be used as a replacement (i plan to test that).

What I wanted to point with that is that the future of documentation format is far from clearn. Still, this does not mean we should not work towards addressing it. There are lots of steps that would apply regardless which formats will gain more traction.

[dougthor42]

I would say that core black should not have an opinion on docstring style (reST, NumpyDoc, Google, etc.).

If anything, docstring style adjustments could be an optional plugin/add-on or even an entirely separate package. black-docstrings or something like that.

Note that there's already blacken-docs which will format python code blocks within docstrings.

[dimaqq]

Using both rst and md with Sphinx (recommonmark) and without Sphinx, I'll second this:

black should not touch any markup in the docstring.

[mgoldey]

Note that there are situations where an opinion inside black would be helpful which AFAIK are not discussed above. For instance, when transitioning from 2-space to 4-space code, black indents the first part of a docstring, but not any subsequent lines, so jagged code as follows is possible:

Given

def some_functions(args):
  """ this is a test
  of some long docstrings
  which go into multiple lines
  """
  pass

black then produces

def some_functions(args):
    """ this is a test
  of some long docstrings
  which go into multiple lines
  """
    pass

This might be a separate concern.

[ichard26]

That has been fixed on master already:

(black) richard-26@ubuntu-laptop:~/programming/black$ black test.py --diff --color
--- test.py 2020-06-15 21:12:39.234785 +0000
+++ test.py 2020-06-15 21:12:48.966120 +0000
@@ -1,6 +1,6 @@
 def some_functions(args):
-  """ this is a test
-  of some long docstrings
-  which go into multiple lines
-  """
-  pass
+    """this is a test
+    of some long docstrings
+    which go into multiple lines
+    """
+    pass
would reformat test.py
All done! ✨ 🍰 ✨
1 file would be reformatted.

[kuwv]

OK, next topic. Is should Black have an opinion on parameter documentation?

E.g.

def func(param1, param2):
    """
    This is a reST style.

    :param param1: this is a first param
    :param param2: this is a second param
    :returns: this is a description of what is returned
    :raises keyError: raises an exception
    """
    pass

More styles here: https://stackoverflow.com/a/24385103

There are three separate standards.

Restructured
Numpy
Google

Using black shouldn't decide what integrations you're able to use.

[JelleZijlstra]

Black now has plenty opinions about docstrings (#1053) and this issue is long enough. If you feel that Black has the wrong opinion, or that it should have opinions about more aspects of docstrings, feel free to open a new issue.

[timokau]

Should we re-open https://github.com/psf/black/issues/745 then? The discussion was supposed to be moved here, but as far as I see no decision has been reached yet.