Yapf: Please manage commas

[bukzor]

I expect yapf, as a style-normalizer, to handle the trailing comma issue for me.
Some writers put it always, others never, and yapf should make the code look the same regardless of who wrote it.

``` #!python
a_very_long_function_name(
long_argument_name_1=1,
long_argument_name_2=2,
long_argument_name_3=3,
long_argument_name_4=4,
)

a_very_long_function_name(long_argument_name_1=1,
long_argument_name_2=2,
long_argument_name_3=3,
long_argument_name_4=4)

Desired output: (in all three cases)

``` #!python
a_very_long_function_name(
    long_argument_name_1=1,
    long_argument_name_2=2,
    long_argument_name_3=3,
    long_argument_name_4=4,
)

a_very_long_function_name(
    long_argument_name_1=1,
    long_argument_name_2=2,
    long_argument_name_3=3,
    long_argument_name_4=4,
)

Actual output: (note trailing comma missing in second case)

$ yapf --style facebook ./demo.py
a_very_long_function_name(
    long_argument_name_1=1,
    long_argument_name_2=2,
    long_argument_name_3=3,
    long_argument_name_4=4,
)

a_very_long_function_name(
    long_argument_name_1=1,
    long_argument_name_2=2,
    long_argument_name_3=3,
    long_argument_name_4=4
)

In this case I have to hunt down missing trailing commas and fix them by hand, which is very similar to the problem yapf was made to solve -- often in code reviews you'll see junior devs told "trailing commas are nice".

[mbra]

Also it would be awesome if this played nicely with SPLIT_ARGUMENTS_WHEN_COMMA_TERMINATED.
When I add elements to an unsplit line, I'd like the trailing comma to be recognized by it.

Input:

function(x=aaaaaaaaaaaaaaaaaaaaaaa, y=bbbbbbbbbbbbbbbbbbbbbbbb, z=cccccccccccccccccccc)

function(aaaaaaaaaaaaaaaaaaaaaaa, bbbbbbbbbbbbbbbbbbbbbbbb, cccccccccccccccccccc)

[aaaaaaaaaaaaaaaaaaaaaaa, bbbbbbbbbbbbbbbbbbbbbbbbbbbb, cccccccccccccccccccccccc]

{'aaaaaaaaaaaaaaaaaaaaaaa': 'x', 'bbbbbbbbbbbbbbbbbbbbbb': 'y', 'cccccccccccccccccc': 'z'}

Current Output:

function(
    x=aaaaaaaaaaaaaaaaaaaaaaa,
    y=bbbbbbbbbbbbbbbbbbbbbbbb,
    z=cccccccccccccccccccc
)

function(
    aaaaaaaaaaaaaaaaaaaaaaa, bbbbbbbbbbbbbbbbbbbbbbbbbbbb,
    ccccccccccccccccccccccccc
)

[
    aaaaaaaaaaaaaaaaaaaaaaa, bbbbbbbbbbbbbbbbbbbbbbbbbbbb,
    cccccccccccccccccccccccc
]

{
    'aaaaaaaaaaaaaaaaaaaaaaa': 'x',
    'bbbbbbbbbbbbbbbbbbbbbbbb': 'y',
    'cccccccccccccccccccc': 'z'
}

Should be:

function(
    x=aaaaaaaaaaaaaaaaaaaaaaa,
    y=bbbbbbbbbbbbbbbbbbbbbbbb,
    z=cccccccccccccccccccc,
)

function(
    aaaaaaaaaaaaaaaaaaaaaaa,
    bbbbbbbbbbbbbbbbbbbbbbbbbbbb,
    ccccccccccccccccccccccccc,
)

[
    aaaaaaaaaaaaaaaaaaaaaaa,
    bbbbbbbbbbbbbbbbbbbbbbbbbbbb,
    cccccccccccccccccccccccc,
]

{
    'aaaaaaaaaaaaaaaaaaaaaaa': 'x',
    'bbbbbbbbbbbbbbbbbbbbbbbb': 'y',
    'cccccccccccccccccccc': 'z',
}

I realize that adding trailing commas is a little harder in function calls, since with * and ** expansions trailing commas trigger a SyntaxError. But I feel like this is also an argument for handling it explcitly, since for those cases manually adding a trailing comma to get 1 element per line is not only extra effort, but will be really annoying.

I'd like to configure my editor to automatically format the file when saving, running pylint directly after the format, which would inevitably trigger a pylint error or require me to indent the function call manually.

Writing this i realize that with this input:

function(aaaaaaaaaaaaaaaaaaaaaaa, bbbbbbbbbbbbbbbbbbbbbbbb, cccccccccccccccccccc, *args)

This is the current output:

function(
    aaaaaaaaaaaaaaaaaaaaaaa, bbbbbbbbbbbbbbbbbbbbbbbb, cccccccccccccccccccc, *
    args
)

Which looks just wrong, it really should be:

function( 
    aaaaaaaaaaaaaaaaaaaaaaa,
    bbbbbbbbbbbbbbbbbbbbbbbb,
    cccccccccccccccccccc,
    *args
)

With this, I also get that it's not as easy as just making SPLIT_ARGUMENTS_WHEN_COMMA_TERMINATED see and add trailing comma since there is no trailing comma for the * and ** expansion cases. :-(

I took a quick look at the code to see if I could do this myself, but I fear I'll have to do some more digging before I get whats going on in there.

[gwelymernans]

In general, I'm against modifying the token stream. I did it before and it's fraught with problems. If someone wants to submit a patch, I'll consider it. But it'll have to be contained behind a style knob and come with a bevy of tests...

[asottile]

for those looking for a battle-tested tool which modifies the token stream to add commas (with options for different target python versions) I've written add-trailing-comma