Php-parser: [feature] implement trailingComments on nodes

[ichiriac]

Related to #189 issue, the parser should be able to extract comments on trailingComments in order to better represent their position on the document.

Sample use case :
https://astexplorer.net/#/gist/2674d53a98c68e273b96adaa99047e61/5c6bbdbfad0d9996203e56dcfae8659ec593fc57

[alexander-akait]

@ichiriac looks we need implement also leadingComments

[alexander-akait]

@ichiriac can we mark it as important and try to implement it faster, this is one of the main problems why we cannot release a stable version :disappointed:

[ichiriac]

Ok, that's the next fix, coming soon

[alexander-akait]

Thanks

[ichiriac]

I note sure I want to exactly reproduce Babylon behavior, the same comment can be on trailing & leading comments - in the example above, the I will not provide the red line. A node should in principle be on only one place :

[ichiriac]

here another strange example of leading behavior - I think it's like this because it's a simpler implementation :

[ichiriac]

Hi @evilebottnawi, it's almost finished, remains a cosmetic fix. You can start now, what you need to do is just prepend leadingComments, and append trailingComments.

I will need to implement #170, #196, #188 (Stmt_Expression) for edge cases

Here how it woks :

   /**
     * TOKENS :
     * node1 commentA token commmentB node2 commentC token commentD node3 commentE token
     *
     * AST :
     * structure:S1 [
     *    left: node1 ( trail: commentA ),
     *    right: structure:S2 [
     *       node2 (lead: commentB, trail: commentC),
     *       node3 (lead: commentD)
     *    ],
     *    trail: commentE
     * ]
     *
     * Algorithm :
     *
     * Attach the last comments on parent of current node
     * If a new node is started and the parent has a trailing comment
     * the move it on previous node
     *
     * start S2
     * start node1
     * consume node1 & set commentA as trailingComment on S2
     * start S2
     * S1 has a trailingComment, attach it on node1
     * ...
     * NOTE : As the trailingComment Behavior depends on AST, it will be build on
     * the AST layer - last child node will keep it's trailingComment nodes
     */

[ichiriac]

It's harder than I though, actual structure is the following :

    /* AST :
     * structure:S1 [
     *    left: node1 ( trail: commentA ),
     *    right: structure:S2 [
     *       // IN SOME CASES commentB is eaten by trail property on node1
     *       node2 (lead: commentB, trail: commentC),
     *       node3 (lead: commentD, trail: commentE)
     *    ]
     * ]
     */

[alexander-akait]

@ichiriac thanks for work, looks it is really not simple. Try to update to latest version and something wrong with comments printing, i will investigate tomorrow and leave feedback, maybe problem on our side.

[alexander-akait]

@ichiriac something wrong with comments after introducing ExpressionStatement. Some comments attach to ExpressionStatement.
Before update:

$data = [
    'value',
    'value',

    // a comment
    'value',
    'value'
];

After

$data = ['value', 'value', 'value', 'value']// a comment;

Same for closure.
Before:

$noArgs_longVars = function () use (
            $longVar1,
            $longerVar2,
            $muchLongerVar3
        ) {
            // Body
        };

After:

$noArgs_longVars = function () use (
            $longVar1,
            $longerVar2,
            $muchLongerVar3
        ) {}// Body;

Same for anonymous class.
Before

$instance = new class extends \\Foo implements
    \\ArrayAccess,
    \\Countable,
    \\Serializable
{
    // Class content
};

After:

$instance = new class extends \\Foo implements
    \\ArrayAccess,
    \\Countable,
    \\Serializable
{
}// Class content;

Also affected assign, propertylookup, ternary and other (not all). I can't find any lines to related this in commit. Maybe problem on our side, but in many places we use same logic as in prettier (with babylon) and before introducing ExpressionStatement all comments ouput in right places.

[ichiriac]

The error in the first example should come from your code, the 3rd item should contain the leading comment, or the 2nd the trailling comment. In principle the expression does not have any comments.

On next examples, the body node does not capture the comment, here your output is correct as the comment is attached as trailling comment at the wrong place.

This part is not yet implemented, I'll work on it soon because the tralling or leading status depends on tokens between 2 nodes on the same level, but actually the code have no idea on what level it's processing tokens.

[alexander-akait]

@ichiriac strange what it is works very good before introducing ExpressionStatement :confused:

[alexander-akait]

@ichiriac problem was on our side with comments, only one problem i find with ExpressionStatement

(call());
call();

generate difference ast, it is bug. In both cases it should be ExpressionStatement.

Looks:

$var->foo;
var['foo'];
var::$property;

Also doesn't generate ExpressionStatement, but should.

[alexander-akait]

@ichiriac Can you finish this feature? You have more knowledge and you do it better and faster, thanks!

[ichiriac]

released in 3.0.0-prerelease.8