Typescript: Suggestion: Generate doc comments in emitted JS

[RyanCavanaugh]

Support some level of automatically generating doc comments for non-TypeScript JavaScript consumers.

Need more details on what exactly people would like to see generated.

[jvilk]

Most importantly, for me: Propagate type information into existing JSDoc comments. This would allow me to use existing JSDoc documentation generators to produce nice looking HTML documentation for my TypeScript libraries without respecifying type signatures in JSDoc comments.

(Of course, constructing a mapping between TypeScript types and JSDoc types might end up being nontrivial, especially where interface types are concerned.)

[jmatthiesen]

Making sure I understand you correctly - you'd like to see the type information from TypeScript persisted into JSDoc comments in the generated .js files from your TypeScript, right? Can you give an example of a documentation generator you currently use?

[jvilk]

you'd like to see the type information from TypeScript persisted into JSDoc comments in the generated .js files from your TypeScript, right?

Correct.

Can you give an example of a documentation generator you currently use?

I am not using a documentation generator at the moment (because it would be infeasible/frustrating to maintain type information in two locations as the code evolves), but I would like to use something like JSDoc: http://usejsdoc.org/

[sebastian-lenz]

I've created a documentation generator that can directly parse *.ts files and that understands TypeScript elements like classes or types. Maybe you would like to check it out:
https://github.com/sebastian-lenz/typedoc

[jvilk]

Your tool looks really nice! I'll have to see how it holds up to my code.

I figure it would be more elegant to avoid needing to maintain a separate project if we could find a decent mapping from TypeScript types and JSDoc annotations, but maybe that is a rathole in-and-of-itself.

[fsoikin]

We have a similar tool, but with a bit broader scope. It uses a template engine to generate output, which makes it perfectly usable for generating docs. Granted, there is more work for you (have to write the templates), but then you can make the output nicely customized.

Check it out: http://github.com/erecruit/TsT

[herrernst]

As the issues on Codeplex are now closed, I'd like to link to the JSDoc request there: http://typescript.codeplex.com/workitem/1596
IMHO, JSDoc annotated output would not only be good for documentation generators but also for other tools.

[mixtur]

+1 to herrernst
It would be nice to have JSDoc in emitted code, not because it would be easier to read later, but because such code can be feed to Google Closure compiler.
Generating already optimized code is another alternative.
Sorry for bad English.

[electricessence]

+1

[ghost]

Yup, annotations for the google closure compiler are a must for me. Having to do it manually really limits the advantages of typing. If you get your manual declarations incorrect, it can lead to incorrectly optimized results.

As far as the "needs proposal" tag goes.. do this please :) -- https://developers.google.com/closure/compiler/docs/js-for-compiler

[mirhagk]

In addition to the JSDoc comments, it may be worth looking into also supporting the exports and externs for the Google Closure Compiler
https://developers.google.com/closure/compiler/docs/api-tutorial3

EDIT: Emitting export and externs for Google Closure is mentioned in #8

[zzo]

+1 for closure-style type comment annoataions

[mindplay-dk]

I believe this feature would weigh in _huge_, as far as convincing established JS projects to "upgrade" their codebases to Typescript, since, effectively, this would eliminate the need to hand-write JSDoc annotations, while resulting in JS code (for plain JS consumers) of the same quality - or higher, for that matter, since Typescript annotations as well as code all gets checked at compile-time. In other words, this would save time _and_ increase quality.

As things stand, without this feature, some JS project maintainers may well hold back, because they don't want to lose their establish plain JS consumer base, a lot of whom depend on annotations for IDE support.

(of course, IMO they should all "upgrade" to Typescript, but I'm obviously opinionated ;-)

[evil-shrike]

+1 for emitting jsdoc tags with type info as much as possible.

[hraban]

I'd like to take a jab at this. Can somebody from the TS project weigh in on the expected amount of work this would require? From an outsider perspective this looks suspiciously easy.

Do you foresee any major roadblocks? Or do you expect this to be as easy as it looks (slightly amend the last "code generating" compiler phase)?

[CyrusNajmabadi]

Hey @hraban, this should not be too much work. You'd basically just need to:

1. Add a compiler option for this.
2. Fix up the emitter to respect the option.
3. The Emitter should likely generate this information for everything we emit, and it should either augment an existing comment we are writing out, or it should add a new comment if none already exists.

[TobiaszCudnik]

There's also the difference with nullability. For the full closure compiler
support you may want to infere the assertions and if conditIons.

On Friday, April 17, 2015, CyrusNajmabadi [email protected] wrote:

Hey @hraban https://github.com/hraban, this should not be too much
work. You'd basically just need to:

1. Add a compiler option for this.
2. Fix up the emitter to respect the option.
3. The Emitter should likely generate this information for everything
   we emit, and it should either augment an existing comment we are writing
   out, or it should add a new comment if none already exists.

—
Reply to this email directly or view it on GitHub
https://github.com/Microsoft/TypeScript/issues/10#issuecomment-94052481.

Tobiasz Cudnik
Software Developer @ Voiceworks
http://voiceworks.com

[mhegazy]

@TobiaszCudnik I would consider this a different feature. I would also argue this has to be only supported for explicit types only not infered ones.

[danquirk]

Note IntelliJ's generation of JS Doc comments includes information about inferred types (in particular, inferred function return types).

[mhegazy]

Doing only declared types allows to implement this as a syntactic transformation, which is a. simple, and b. works for single file emit. It also allows users to manage what there public API (assuming that this is what it is used for).

[hraban]

Hey guys! I'm sorry but I am absolutely swamped at the moment. It's in the back of my mind, but on hold for a while. Definitely interested to see whatever happens once this gets picked up, though!

[ustims]

This is very outdated, but probably could be helpful: https://github.com/evanw/typescript-closure-compiler

[hraban]

Hello everyone,

I've been looking around the relevant code recently and going through the comments here.

I agree with the "we should extend existing JSDoc comments" point, _eventually_. It makes the most sense from an integration-with-current-tools perspective, and it would be a nice selling point, as the resulting JS would then be JSDoc consumable with automatic type annotations from TS.

This leads to situations like:

/**
 * Determine the logarithm of a number to a specific base
 *
 * @param b The base of the logarithm
 * @param {weirdType} g The number to determine the logarithm of
 * @return The logarithm of g in base b
 */
function logB(b: number, g: number): number {
    return Math.log(g) / Math.log(b);
}

Where, ideally, you would want this as the output:

/**
 * Determine the logarithm of a number to a specific base
 *
 * @param {number} b The base of the logarithm
 * @param {weirdType} g The number to determine the logarithm of
 * @return {number} The logarithm of g in base b
 */
function logB(b, g) {
    return Math.log(g) / Math.log(b);
}

In that case, we'd need not only a proper JSDoc parser, but also a JSDoc serializer.

Right now, JSDoc is not even parsed, at all, in .ts files, only in .js. We can enable it for .ts files, but that still leaves the problem of "changing" the comment before output; the comments are just copied verbatim from the source file, not rebuilt from a "jsdoc AST" as it were (even though they are properly parsed that way).

So this would seem non-trivial. What are your thoughts on this, so far?

There are a few ways around this, just to "get started", some quick and dirty solutions to get more than what we have right now, with less effort than the above. One option is to completely ignore nodes with existing JSDoc comments, hidden behind an appropriate flag, e.g. --generate-missing-JSDoc. It's not pretty (disincentivizes creating JSDoc by users), but it's a start. Ideally this will get us closer to a full solution.

What do you think? Did I misunderstand something from the code, perhaps? I'm curious to hear what people who are more familiar with the code structure have to say about it.

Regards

Hraban

[mhegazy]

@hraban you are right, we will need to 1. parse jsdoc in .ts files, 2. Emit the jsdoc correctly from the AST and 3. Augment the jsdoc with type information if the jsdoc already exist.

1, is fairly trivial and so is 2. 3 needs some synthetic nodes to be created, but should not be thst complicated either (I belive it should be done as a syntactic transformation and only for explicit type annotations and not infered types).

If this is something you would like to contribute, I would be happy to help out as much as possible.

[DanielRosenwasser]

@mhegazy, isn't @aozgaa working on something similar?

[hraban]

@DanielRosenwasser Ouh, if that's the case then it's good to know now :) I sent Arthur an e-mail to ask him about it.

@mhegazy:

1. If all you want is information that is used by the compiler, then this is indeed trivial. Remove the if-guard at if (isJavaScript(fileName)) { addJSDocComments() } in parser.ts, and you have augmented the AST with JSDoc.
2. However, addJSDocComments is incomplete. It ignores JSDoc elements that are not param, return, returns, type or template. If you use the parsed AST to serialize a JSDoc comment, you will lose all other tags. I think (?). Is that okay?
3. I'm not religious about it, just fmi: why do we not want inferred types?

Thanks for offering help, I'm exploring what needs to be done for this and I might take you up on that offer. :)

[mhegazy]

@aozgaa is working on jsdoc tooling in the language service. so this is up for grabs.

@hraban

However, addJSDocComments is incomplete. It ignores JSDoc elements that are not param, return, returns, type or template. If you use the parsed AST to serialize a JSDoc comment, you will lose all other tags. I think (?). Is that okay?

I do not know, what other tags we need? do you have a list in mind? i think we should be adding them as we go, or just add a catch all generic tag that would be available in the tree but ignored by tooling cause they do not understand them. @CyrusNajmabadi would have better ideas here.

I'm not religious about it, just fmi: why do we not want inferred types?

1. simplicity, no checker/ type involvement. 2. fits what we have been doing so far, of no type-directed emit. and 3. works when you are using ts.transpile, i.e. as a single file syntactic transformation and 4. (more of me pontificating) i think if you did not bother to put a type annotation, it should not be in your API, if you want it to be, may be you should think about it and explicitly specify it.

[hraban]

re: the other tags: if we make a whitelist of tags we parse to AST and rebuild the final JSDoc from that AST, the rest is lost in the output. This effectively cripples the JSDoc dialect (which is now 100%) to only those tags; the rest will be removed by tsc. This doesn't sound ideal?

[mhegazy]

that is why i am saying parse them, keep them in the AST, but not report errors for them for instance.

[hraban]

Understood.

I think your reason nr 4 is pretty sensible, considering sometimes automatic types are inferred quite aggressively. E.g. let vals = [{value: 3}, {value: 4}].map(x => x.value) being number[] could just be a coincidence, and maybe a future value will be "foo".

Roadmap:

1. Parse JSDoc in .ts files
2. Create a wild-card JSDoc node that holds unknown directives
3. Rebuild JSDoc from the parsed AST (and verify that it's semantically equivalent)
4. Add type info to JSDoc AST where missing from original JSDoc but specified explicitly in type annotations.

Everyone agree?

How do I create unit tests to test the " syntactical transformation" part?

[mhegazy]

@CyrusNajmabadi any objections/suggestions

we also need a new command line arguments for generating documentation in the output.

the test cases for ts.transpile are in here: https://github.com/Microsoft/TypeScript/blob/master/tests/cases/unittests/transpile.ts

[CyrusNajmabadi]

@mhegazy

1, is fairly trivial and so is 2. 3 needs some synthetic nodes to be created, but should not be thst complicated either (I belive it should be done as a syntactic transformation and only for explicit type annotations and not infered types).

I agree with '1' and '2'. I'm not sure that '3' needs us to go so far as producing Synthetic nodes or anything that complicated. I would just take the AST for the DocComment and emit that with some specialized emit logic. It would basically emit the comment mostly as is, with some specialized logic around dealing with certain tags. For example, for an @param tag, we would:

1. See if it already had a {...} type expression. If so, we wouldn't do anything special.
2. If now we would just try to find the corresponding parameter for that @param tag.
3. We'd then find out its type.
4. Finally we would emit that type using JSDoc type exprssion notation in the doc comment we were producing.

I'm not sure I see the need for any sort of synthetic node stuff to handle this.

Note: It's also not that necessary to include the JSDoc AST in the full AST. Instead, you could just parse the JSDoc AST on-demand in certain places in the emitter. This is what we do, for example, when classifying .ts files. When we encounter a .jsdoc comment, we parse it then and there and then do not keep it around afterwards.

[CyrusNajmabadi]

@hraban

However, addJSDocComments is incomplete. It ignores JSDoc elements that are not param , return , returns , type or template . If you use the parsed AST to serialize a JSDoc comment, you will lose all other tags. I think (?). Is that okay?

This is not quite accurate. The JSDoc parser doesn't ignore or lose JSDoc elements. It just doesn't create specialized nodes for JSDoc elements that it doesn't consider special. It will always at least create a tag no matter what you end up putting after the @ . This is handled in the handleUnknownTag branch of this code:

                    let tag = handleTag(atToken, tagName) || handleUnknownTag(atToken, tagName);
                    addTag(tag);
                }

So you could still preserve tags without any problem as they will be in the AST.

Create a wild-card JSDoc node that holds unknown directives

Already handled with the raw SyntaxKind.JSDocTag kind :)

I hope that helps!

[evil-shrike]

Let me add some thoughts.
Generating and/or updating jsdoc for emitted js isn't enough. There are interfaces in TS which should become @typedef doclets in jsdoc comments in js.
BTW currently if I add a @typedef doclet before an interface in TS (even not in .d.ts) it will be stripped out in emitted js.

[hraban]

@evil-shrike what are your thoughts on making that a new issue altogether? We could split this up, and make this issue just about annotating e.g. function parameter types and variable types. Then we grow from there. What do you think?

[evil-shrike]

I don't mind, but my point is related to "annotating function params".
It's common practice to use interfaces for params:

export class Area extends lang.Observable {
    /**
     * @constructs Area
     * @extends Observable
     * @param options
     */
    constructor (options: Area.Options) {}
}
module Area {
    export interface Options {
        title?: string;
        hidden?: boolean;
    }
}

What type for options params should be specified?
As it's typescript code it's logically to use Area.Options but it'll become meaningless in js.
Currently I have to repeat all Area.Options properties in jsdoc:

    /**
     * @constructs Area
     * @extends Observable
     * @param {Object} options
     * @param {String} [options.title] Area title 
     * @param {Boolean} [options.hidden] true to create a hidden area (can be shown later with `show`)
     */
    constructor (options: Area.Options) {}

If this type is being used in several places I'll have to repeat the comment every time. And still I have the interface undocumented in TS, so I have to repeat doc one more time for the interface:

    export interface Options {
        /** Area title */
        title?: string;
        /** true to create a hidden area (can be shown later with `show`) */
        hidden?: boolean;
    }

Too much work.

As I understand you're considering generating jsdoc comments as if no interface exists (as in the example above, without introducing a new entity as @typedoc). Right? This's better that nothing indeed. But then please consider using documentation from TS's interface properties.

Also it's unclear for me what correct behavior of the compiler would be in a case when I put TS type information for a @param in TS-code:

@param [Area.Options] options

Replacing with in-place flat structure (options.title, options.hidden) would be incorrect as I could put @typedef somewhere in js on my own and I'll lost the relation with typedef.

I believe that the problem here is that it's not stated clearly what jsdoc comments in TS code is (are?). Is it for TS or for JS only? If it's for TS then it should become part of language spec.

p.s. as for issue splitting, please feel free to move any my comments or ignore them ). I'm not sure I can correctly specify a new issue.

[levinmr]

+1 for emitting closure compiler annotations

[eggers]

+1

[kevin-smets]

Any progress for this issue? Or is there a recommended way for now to take a stab at generating docs? (looking into typedoc for now).

We're investigating the migration of some big projects to TypeScript and obviously, documentation is a big part of this, so I'm just wondering. Thanks!

[RyanCavanaugh]

This isn't in our near-term roadmap.

I'm not sure what the state of the art in other tools that might able to do this is, either.

[christru]

+1

[jvilk]

The best TypeScript doc generator I've found is TypeDoc, but it doesn't have nearly the same level of community support as JsDoc. TypeDoc is maintained by one person, who may be a bit overwhelmed with the level of interest given the rate of unanswered bug reports.

The rest of the TypeScript documentation tools I've found are orphaned, as they either:

• Forked the TS compiler, which results in a lot of maintenance work as internal compiler interfaces evolve / new language features are added.
• Wrote/use a custom TS parser, which results in maintenance work as the language syntax evolves.

Assuming JsDoc has reasonably complete ES6 support, it seems like it would make the most sense for a TypeScript documentation tool to output JsDoc to take advantage of all of the tooling centered around that format. And if that's enough to get reasonable Closure compiler support, I'm sure many would be quite happy!

[rick-agilifly]

I like the fact that TypeDoc supports a good number of the Typescript features and we use it within our own projects. That said, I think it would behoove (dating myself here) Microsoft to possibly assist Sebastian in updating Typedoc or at least help integrate it with JSDoc ( perhaps as gulp components for both sides ). As Typescript continues to gain traction I think it will show the JS community at larger that this is a worthwhile tool and needs an investment in the tooling ( much like has already happened in gulp)

[Oceanswave]

+1

[alexeagle]

wow issue #10 so few digits

I see comments here about emitting docs for Closure Compiler, without much mention that for complex types the translation between the two type systems is non-trivial and lossy. Now that TypeScript will have nullability it's improved a bit.

@martine on our team works on Tsickle which produces closure-compiler-compatible TypeScript code that can down-level to ES6 with goog.module module syntax, and has a (presently incomplete) typed mode to carry forward the type information. https://github.com/angular/tsickle

[evmar]

Related: https://github.com/Microsoft/TypeScript/issues/7393 is the smaller feature request, specifically to expose the compiler's JSDoc parser API for tools like documentation extractors.

Tools like TypeDoc currently must have their own separate parser for JSDoc on top of the TS API:
https://github.com/TypeStrong/typedoc/blob/2e855cf8c62c7813ac62cb1ef911c9a0c5e034c2/src/lib/converter/factories/comment.ts#L78

[claytonsilva]

+1

[develar]

As TypeDoc has a lot of errors and cannot compile fully correct project, ts-jsdoc was implemented. (https://github.com/develar/ts2jsdoc). It allows you to use any jsdoc template. And reuse all features that jsdoc has. No need another special tool for TypeScript (e.g. JSDoc supports interfaces).

generated jsdoc annotated js file

generated docs

Only public TS API is used, any compilable project is supported. Currently, tested only for node modules. And function (callback) types are not supported (planned).

[JasonKleban]

@RyanCavanaugh - Can I ask what is the current state of this? Preserving type information through to jsdoc-consuming documentation generators is a sticking point for my team's adoption of Typescript in a environment that requires supporting non-typescript project groups.

[kimamula]

I am trying to solve this with custom transformers, as it looks like that the feature exists exactly for solving this kind of problem.

However, I cannot find how to mutate ts.JSDoc with custom transformers.
There's no method like ts.updateJSDoc().
ts.updateMethod(), for example, does not accept a parameter of ts.JSDoc type.
https://github.com/Microsoft/TypeScript/blob/1db4f96fd14fc3de5ae3704e925afd6474cfb8f5/src/compiler/factory.ts#L582
I also tried to directly mutate ts.JSDoc which is obtained by iterating over children of ts.MethodDeclaration, which had no effect in the emitted JavaScript.

Is it possible to mutate ts.JSDoc with custom transformers?

[aoberoi]

I would really love to see JSDoc comments in the emitted .js. I consider updating the comments in .ts files a "nice to have" feature, but not truly necessary, since documentation generators could be run on the emitted .js instead.

I believe solving this issue would really further the TypeScript team's story about being able to adopt the language incrementally and its compatibility with the existing JavaScript ecosystem (as opposed to forking the community and building separate tools).

[DrMiaow]

I would be using more TypeScript, but I need the full functionality of JSDoc. I am actually in the process of converting one of my experimental projects from TypeScript to ES6 simply because I need JSDoc documentation more than I need the features of TypeScript.

[leidegre]

I would like to add that the Google Closure Compiler might be able to leverage such JSDoc tags to produce a smaller file size. Since I know the topic of minification has been discussed before with respect to the superior type information available to TypeScript, this would seem like a two birds with one stone opportunity.