Pycodestyle: E126 on `with` statement line continuation

[darkfeline]

E126 (continuation line over-indented for hanging indent) is reported on what I think should be correct formatting on line continuation in multiline with statements:

with open('foo') as foo, \
     open('bar') as bar, \
     open('baz') as baz:
    pass

PEP8 doesn't specify this explicitly, but I believe this is a perfectly reasonable and natural way to indent such a statement. Also, IDLE automatically indents this the same way.

[sigmavirus24]

PEP8 does specify how _hanging-indents_ should be treated and pep8.py does exactly what the document says it should.

Also, IDLE automatically indents this the same way.

Editor behaviour is not what defines the rules in the document or the behaviour of this tool.

[darkfeline]

Actually, upon careful reading of PEP8, it says that "The 4-space rule is optional for continuation lines" and "Hanging indents _may_ be indented to other than 4 spaces".

The point I was trying to make was that, for example with while statements, PEP8 specifically allows for aligning with the opening delimiter:

while (foo == foo and
       bar == bar and
       baz == baz):
    pass

with statements cannot take parentheses for implicit line continuation, so the only option, given pep8 (this program) as it is now, is a double indent, which looks ugly:

with open('foo') as foo, \
        open('bar') as bar:
    pass

or simulate a "hanging indent", as described in PEP8 (see the footnote on the PEP8 page) (even uglier):

with \
        open('foo') as foo, \
        open('bar') as bar:
    pass

What I was requesting is simulating the alignment with the opening delimiter for with statements:

with open('foo') as foo, \
     open('bar') as bar:
    pass

However, your post made me go back and re-examine PEP8, and I think that pep8's checking of continuation lines should be even looser, because according to PEP8, any level of indentation is acceptable, as long as it isn't indistinguishable from the indentation of a following suite:

if (foo == foo and
    foo == bar):  # same as below; bad
    pass

Futhermore, while I agree that editor behaviour is not prescriptive, IDLE is officially bundled with Python and its auto-indentation otherwise follows PEP8 accurately, so I believe its behaviour should be given some weight.

[sigmavirus24]

@darkfeline the section you're referring to says:

Use 4 spaces per indentation level.

And further the initial examples all show that pep8's current behaviour is correct. It says you may indent to other indentation levels but that doesn't mean the former examples are to be thrown out. How we interpret that means you are free to use --ignore=E126 to indent amounts that are not a multiple of 4 or visual.

Your examples that use opening delimiters (namely parentheses) work because they are allowed to use delimiters like that. A with_statement has no such allowance in the grammar of the language. Further if it did your statement would look like:

with (open('file0') as file0,
      open('file1') as file1):
    pass

Which means that to simulate that behaviour you would _actually_ want to indent like so:

with open('file0') as file0, \
      open('file1') as file1:
    pass

That _would_ be visual if the grammar allowed for parentheses, but it doesn't so that is a moot argument. In the absence of that, it says to indent a multiple of 4. Whether IDLE is officially bundled with Python or not is not a concern and how it treats other indentation is not either.

As I see it you have a few options:

• Fork pep8 to allow your behaviour and use your fork
• Indent a multiple of 4 spaces
• Ignore E127

[darkfeline]

Further if it did your statement would look like:

with (open('file0') as file0,
      open('file1') as file1):
    pass

Which means that to simulate that behaviour you would actually want to indent like so:

with open('file0') as file0, \
      open('file1') as file1:
    pass

I'm a little surprised by how you don't see that the point is that the two opens are aligned in the first example and not in the second. I _actually_ want to indent like so:

with open('file0') as file0, \
     open('file1') as file1:
    pass

Notice the aligned expressions (the open function calls in this example), this is visually pleasing.

And further the initial examples all show that pep8's current behaviour is correct. It says you may indent to other indentation levels but that doesn't mean the former examples are to be thrown out.

In that case shouldn't E126 instead be a W1 warning? W191 (indentation with tabs) is a warning, while it is specifically banned by PEP8. On the flip side, non-4-space continuation lines is explicitly allowed by PEP8 (though, as you say, the examples all follow 4-space indents), yet it is reported as an error.

I can see why you would want to keep pep8's current behavior, but I think that if pep8 isn't going to ignore it (which I still think it should), it should at least be reduced to a warning for consistency's sake.

I would like one more chance to convince you that removing E126 is consistent with the PEP8 style guide.

Specifically, PEP8 says the following:

Use 4 spaces per indentation level.

Note that this applies to indentation _levels_ (for suites), and _not_ continuation lines, which is the subject of E126. Continuation lines are described thusly.

Continuation lines should align wrapped elements either vertically using Python's implicit line joining inside parentheses, brackets and braces, or using a hanging indent [5]. When using a hanging indent the following considerations should be applied; there should be no arguments on the first line and further indentation should be used to clearly distinguish itself as a continuation line.

Here we see that there are two options:

1. vertical alignment using implicit line joining in parentheses
2. hanging indent

The first doesn't apply in this situation, so let's look at the second. PEP8 describes a hanging ident as indenting the lines after the first line further than the first line to distinguish it from the first line, and _specifically states_ that the 4-space indentation rule does _not_ apply to _continuation lines_. Again recall that continuation lines are not the same as regular indentation of code in suites.

Thus, the only error PEP8 should report for the indentation of continuation lines is if the continuation line is indented the same amount as the first line, and any other style "errors", if they are kept in pep8, should be reduced to warnings at most.

[sigmavirus24]

I'm a little surprised by how you don't see that the point is that the two opens are aligned in the first example and not in the second.

I do understand that. I'm telling you I disagree. I could feign surprise but I won't.

this is visually pleasing

That argument doesn't fly. Many people, for example, think the following is visually pleasing:

varone    = 1
vartwo    = 2
longervar = 3

But that violates PEP8.

I can see why you would want to keep pep8's current behavior, but I think that if pep8 isn't going to ignore it (which I still think it should), it should at least be reduced to a warning for consistency's sake.

I agree that because of the wording of the document it should be a warning but there are problems with that:

1. The document was worded that way when this check (error) was created.
2. Many people [1] rely on the error and warning codes not changing between non major versions (e.g., 1.5 and 1.6).

So while it would be nice to change this to a warning, we cannot. Or at least, we can't do that until version 2.

specifically states that the 4-space indentation rule does not apply to continuation lines.

Again, were only the document immutable, then this wouldn't be an issue because when the check was written this would have been accounted for.

[darkfeline]

@sigmavirus24
I see, that is very reasonable. Shall this bug be kept open and marked for if/when the next major version of pep8 is released?

[sigmavirus24]

@darkfeline I can't close issues so the answer is _maybe_. That's entirely up to @florentx

[akitada]

PEP 8 was updated and now it has a multiline with-statement example
that is visually aligned with the first item.

So, I think @darkfeline's request is a reasonable and E126 should not be reported on those code.

[sigmavirus24]

@akitada submit a pull request then and we'll see how it is received.

[akitada]

@sigmavirus24 How about just adding E126 to DEFAULT_IGNORE?
I think this issue is similar to #256 and @florentx is proposing to make E121 ignored by default.
Why not add E126 and others reported on PEP 8 examples to thet list?
Any concerns?

[sigmavirus24]

@akitada because E126 is also responsible for checking hanging indents like these if I remember correctly, so it is still valid.

[akitada]

@sigmavirus24 I ran pep8.py over those code but E126 didn't pop up.

$ cat no.py
# Arguments on first line forbidden when not using vertical alignment.
foo = long_function_name(var_one, var_two,
    var_three, var_four)

# Further indentation required as indentation is not distinguishable.
def long_function_name(
    var_one, var_two, var_three,
    var_four):
    print(var_one)
$ python pep8.py no.py
no.py:3:5: E128 continuation line under-indented for visual indent
no.py:6:1: E302 expected 2 blank lines, found 1
no.py:8:5: E125 continuation line with same indent as next logical line

I found another issue with E126, #167 and I think @methane's assertion is correct.

[IanLee1521]

@akitada -- I believe you aren't seeing the E121 / E126 errors pop because your examples are indented by 4 spaces, which is allowed by the tool.

I've proposed PR #360 to add both error codes to the DEFAULT_IGNORE list.

[IanLee1521]

E121 and E126 added to DEFAULT_IGNORE in pull request #360.

[clayg]

So now that pep8 exemplifies 5 space indents on with statement continuations [1] will E126 be updated?

with open('/path/to/some/file/you/want/to/read') as file_1, \
     open('/path/to/some/file/being/written', 'w') as file_2:
    file_2.write(file_1.read())

1. https://www.python.org/dev/peps/pep-0008/#maximum-line-length (near the bottom of the section)

[sigmavirus24]

@clayg I don't understand your question. It's been disabled by default.

[clayg]

@sigmavirus24 yeah! I'd rather it can be turned on but just always do the right thing! Maybe I'm misunderstanding this bug report tho - I was just looking for "with statement line continuation" - I'm not necessarily sure if E126, E127 or E121 are the culprit? The issue as far as I'm concerned is the pep8 tool flagging as an error formatting that is provided in pep8 as exemplary?

I rather like E126/E127/E121 for their original job of dealing with dangling-if's - and in fact I think it does a bang up job there [1] - these checks predates language support for multiple with statement support AFAIK.

But when I turn on all the E's (as I do in most of the projects I work on) it incorrectly forces me to indent my with statements in a manor that's not congruent with pep8's examples for indenting multi-line-with-continuation. I don't think necessarily that the form it enforces is _incorrect_ per say:

with open('/path/to/some/file/you/want/to/read') as file_1, \
        open('/path/to/some/file/being/written', 'w') as file_2:
    file_2.write(file_1.read())

... looks fine to me, but according to the examples in pep8 alignment of the hanging lines on multi-line-with-continuations can also be five spaces:

with open('/path/to/some/file/you/want/to/read') as file_1, \
     open('/path/to/some/file/being/written', 'w') as file_2:
    file_2.write(file_1.read())

^ but that would trigger E121 (different versions of pep8 may have raised E126/E127 in the past?)

I hate that my choice is format multi-line-with-continuation statements differently from the example in pep8 or ignore all errors with multi-line-continuation according to the examples in pep8 by disabling entirely a bunch of E checks that I rather like! :cry:

My question is if the pep8 tool's checks could be updated so that I can I have these E checks turned on without the unfortunate behavior of flagging correctly [2] formatted multi-line-with-continuations?

Perhaps the checks could be updated to exclude/ignore with statements?

Thanks!

1. Well, actually I'm not sure I understand the rules for multi-line-if-continuation, it seems like E121 or E126 should fire for aligned_to_if_to_condition? https://gist.github.com/clayg/98c57584d7476ad70829
2. "correctly" according to the examples in the pep8 document/rules