Flow: Add support for attaching JSDoc comments to AST nodes

[orta]

E.g. With this file:

/** stay out of the Danger zone */
var myVar = "async () => 42;"

/**
 * Hello there
 * @param {String} sure
 */
function myThing(sure: string) {}

Then running bin/flow ast newtests/esdocs/foo.js --pretty

{
  "errors":[],
  "tokens":[],
  "type":"Program",
  "loc":{"source":null,"start":{"line":2,"column":0},"end":{"line":10,"column":1}},
  "range":[35,148],
  "body":[
    {
      "type":"VariableDeclaration",
      "loc":{"source":null,"start":{"line":2,"column":0},"end":{"line":2,"column":29}},
      "range":[35,64],
      "declarations":[
        {
          "type":"VariableDeclarator",
          "loc":{"source":null,"start":{"line":2,"column":4},"end":{"line":2,"column":29}},
          "range":[39,64],
          "id":{
            "type":"Identifier",
            "loc":{"source":null,"start":{"line":2,"column":4},"end":{"line":2,"column":9}},
            "range":[39,44],
            "name":"myVar",
            "typeAnnotation":null,
            "optional":false
          },
          "init":{
            "type":"Literal",
            "loc":{"source":null,"start":{"line":2,"column":12},"end":{"line":2,"column":29}},
            "range":[47,64],
            "value":"async () => 42;",
            "raw":"\"async () => 42;\""
          }
        },
      ],
      "esdoc": {
          "type":"Block",
          "loc":{"source":null,"start":{"line":1,"column":0},"end":{"line":1,"column":34}},
          "range":[0,34],
          "value":"stay out of the Danger zone "
        },
      "kind":"var",
    },
    {
      "type":"FunctionDeclaration",
      "loc":{"source":null,"start":{"line":8,"column":0},"end":{"line":10,"column":1}},
      "range":[113,148],
      "id":{
        "type":"Identifier",
        "loc":{"source":null,"start":{"line":8,"column":9},"end":{"line":8,"column":16}},
        "range":[122,129],
        "name":"myThing",
        "typeAnnotation":null,
        "optional":false
      },
      "params":[
        {
          "type":"Identifier",
          "loc":{"source":null,"start":{"line":8,"column":17},"end":{"line":8,"column":29}},
          "range":[130,142],
          "name":"sure",
          "typeAnnotation":{
            "type":"TypeAnnotation",
            "loc":{"source":null,"start":{"line":8,"column":21},"end":{"line":8,"column":29}},
            "range":[134,142],
            "typeAnnotation":{
              "type":"StringTypeAnnotation",
              "loc":{"source":null,"start":{"line":8,"column":23},"end":{"line":8,"column":29}},
              "range":[136,142]
            }
          },
          "optional":false
        }
      ],
      "body":{
        "type":"BlockStatement",
        "loc":{"source":null,"start":{"line":8,"column":31},"end":{"line":10,"column":1}},
        "range":[144,148],
        "body":[]
      },
      "async":false,
      "generator":false,
      "predicate":null,
      "expression":false,
      "returnType":null,
      "typeParameters":null,
      "esdoc": {
          "type":"Block",
          "loc":{"source":null,"start":{"line":4,"column":0},"end":{"line":7,"column":3}},
          "range":[66,112],
          "value":"*\n * Hello there\n * @param {String} sure\n "
        }
    }
  ],
  "comments":[
    {
      "type":"Block",
      "loc":{"source":null,"start":{"line":1,"column":0},"end":{"line":1,"column":34}},
      "range":[0,34],
      "value":"* stay out of the Danger zone "
    },
    {
      "type":"Block",
      "loc":{"source":null,"start":{"line":4,"column":0},"end":{"line":7,"column":3}},
      "range":[66,112],
      "value":"*\n * Hello there\n * @param {String} sure\n "
    }
  ]
}

Where-in the parser would attach anything matching /** and */ as a part of the syntax tree, which is similar to how Swift and TypeScript (I think) handle passing documentation around.

There are numerous advantages to exposing this, but the best for me is that you can make the developer experience significantly nicer for clients like nuclide/vscode. Especially when people start exposing the API's documentation via auto-complete in flow-typed (pic using viscose's inferrence).

This also then means you can attach JSDoc documentation too the JS libraries in /lib/ making all that documented for everyone too 🎉

[gabelevi]

I wonder what the precedent is for adding docblocks to the AST

Playing around in https://astexplorer.net/

• babylon6 seems to have a leadingComments property for declarations or statements (StatementListItem's I suppose). It seems to include any line or block comment that comes before the StatementListItem.
• esprima with the attachComment option does the same as babylon6

So yeah, I suppose we could do the same pretty easily. It's a bit of a refactor, since it would probably involve messing with the Statement.t type to make it also carry the leading comments.

[ariya]

JFYI, the implementation for this in Esprima is quite short: https://github.com/jquery/esprima/blob/master/src/comment-handler.ts. Perhaps it's useful as a reference 😸

[entropitor]

I think with the implementation of flow-language-server, this really makes sense to support

[jaredly]

Looks like interned comments are now supported? (see https://github.com/facebook/flow/commit/21c1645e134a0c79e09c9b10875c26a06ed7f333 for example)