React-styleguidist: Replace doctrine with another JSDoc parser

[jasonwilliams]

As part of the Webpack 5 investigation it was discovered doctrine uses some node modules which WP5 no longer polyfills by default. Doctrine itself is end of life and no longer supported. It would be good for styleguidist to move off it permanently and replace it with another solution.

Doctrine is used in these places:
https://github.com/styleguidist/react-styleguidist/search?q=doctrine

[poteirard]

I guess jsdoctypeparser would be a good alternative? or maybe https://swc.rs/rustdoc/jsdoc/ would be good for performance?

[jasonwilliams]

@poteirard it could be yeah. Best way is to look at what doctrine is doing and seeing if the same can be done with another parser. The swc one looks like a rust crate, is that on npm?

[sapegin]

Looks like it's also used by react-docgen so replacing it in Styleguidist won't solve the problems with webpack 5? It's still a good idea to replace it though.

Looks like jsdoctypeparser does only half of the job: it parses the types (String[]) but we also need to find JSDoc declarations inside comments in the source code.

[poteirard]

I found https://github.com/syavorsky/comment-parser does the other half of the job. I'm trying to make it work but since it's not working exactly the same as the other I'm struggling a bit to make it work (WIP work: https://github.com/styleguidist/react-styleguidist/commit/7df84e83dd81a12954172334dbbf3fa69fdb02ec)

[sapegin]

This library looks good! I think a better solution would be to extract all the code that parses JSDoc into its own module that would return normalized object with merge synonyms and so on, instead of using these libraries directly in getProps.

[poteirard]

Sorry, not sure if I understood correctly. You mean to modify react-docsgen with these libraries and then use it in react-styleguidist?

[sapegin]

I mean to make a new module inside Styleguidist that would generate usable object with JSDoc data, something like:

function getJsDocTags(code) {
  // magic
  return {
    params: [...],
    returns: 'string',
    // ...
  }
}

The current approach is too verbose and brittle, and it'll be worse with two libraries.

[poteirard]

So you mean in the props-loder.ts I need to replace getProps by a new module which has a getJsDocTags that generates the same output, right?

[sapegin]

Not exactly ;-) we'll keep getProps but most lines you changed in your branch would go to this new module. So I want to encapsulate:

• Parsing JSDoc from the source code (or whatever we do here).
• Parsing each JSDoc declaration.
• Merging aliases, like params and returns.
• Building a typed (meaning we have types for each specific tag not generic types) object that we could use later in getProps.

[poteirard]

Trying again :)
Something I noticed is the optional parameters are different now and all the type keys.
But:

1. I'm not sure how to fix the merge with the description if someone can give me a hand it will be nice. It's hard for me to follow.
2. The react-docgen types are incompatible with the new implementation. So I'm not sure if <ArgumentRenderer/> (line 43) will work as expected if I use the stringifying from the new library . Would it make sense to upgrade react-docgen first given doctrine is no longer maintained?

https://github.com/poteirard/react-styleguidist/tree/replace_doctrine

[sapegin]

Updating react-docgen is always a good idea!

Could you open a pull request from your branch? It would be much easier to check.

[poteirard]

Created: https://github.com/styleguidist/react-styleguidist/pull/1711