Typescript: Include TypeChecker in TransformationContext so that custom transformers can use type information

[kimamula]

Search Terms

customTransformers, custom transformers

Suggestion

Though there are several module bundler plugins that support custom transformers, such as ts-loader, awesome-typescript-loader, parcel-plugin-typescript, rollup-plugin-typescript2, currently some of them cannot accept custom transformers that require TypeChecker.
Including TypeChecker in TransformationContext resolves this issue.

Use Cases

ts-loader, for example, currently supports custom transformers by providing getCustomTransformers option whose type is () => { before?: TransformerFactory[]; after?: TransformerFactory[]; }.
It is not possible to use custom transformers that require TypeChecker such as ts-transformer-keys (it actually requires Program but can be rewritten so that it only requires TypeChecker) with ts-loader.

Though awesome-typescript-loader and rollup-plugin-typescript2 supports such custom transformers
by passing Program or LanguageService, respectively, to the callback corresponding to getCustomTransformers in ts-loader, I found that it is difficult to modify ts-loader's code so that it can also pass Program or LanguageService (or TypeChecker) to the callback.

Examples

Using ts-transformer-keys with ts-loader as described above.

Checklist

My suggestion meets these guidelines:

• [x] This wouldn't be a breaking change in existing TypeScript / JavaScript code
• [x] This wouldn't change the runtime behavior of existing JavaScript code
• [x] This could be implemented without emitting different JS based on the types of the expressions
  
  
  
  • Depends on the behavior of custom transformers to use, but at least TypeScript itself does not need to emit different JS based on the types of the expressions
  
  
• [x] This isn't a runtime feature (e.g. new expression-level syntax)

[mhegazy]

The type checker is intentionally separated from the transformation process. the rational is that two processed do not need to be serialized. e.g. to get the errors in your editor, you do not need to transform the code, and to transform a single file (ts.transformModule) you do not need to type check the whole project.

That said, you can capture the checker from the compiler API. i do not recommend making any type-directed emit transforms though.

[kimamula]

As TypeScript does not provide a functionality to emit JS based on the types of the expressions, implementing custom transformers that depend on TypeChecker is the only way for developers to achieve that functionality.
Therefore, I believe this is one of the most common use cases of custom transformers. Angular also implements and uses such custom transformers.
Providing TypeChecker as a property of TransformationContext is helpful to use such custom transformers.

[typescript-bot]

Automatically closing this issue for housekeeping purposes. The issue labels indicate that it is unactionable at the moment or has already been addressed.

[gfx]

I'd like this issue to be re-opened.

Custom transformers with TypeChecker give us the power to extend the TypeScript language itself.

[dsherret]

How would this be implemented properly? Isn't it possible for the type checker to be out of date with the current state of the AST due to past transformations (ex. another transform plugin running before)?

By the way, here's some existing issues with using the type checker during transformation. This outputs number both times:

console.log(typeChecker.typeToString(typeChecker.getTypeAtLocation(declaration))); // number
declaration = ts.updateVariableDeclaration(declaration, declaration.name, ts.createTypeReferenceNode("Date", undefined), undefined);
console.log(typeChecker.typeToString(typeChecker.getTypeAtLocation(declaration))); // number
console.log(typeChecker.typeToString(typeChecker.getTypeAtLocation(declaration.type!))); // any

Also worth noting that nodes created by a factory function return any:

const declaration = ts.createVariableDeclaration("test", ts.createTypeReferenceNode("Date", undefined));
const newStatement = ts.createVariableStatement(undefined, [declaration]);
sourceFile = ts.updateSourceFileNode(sourceFile, [newStatement], false, undefined);
console.log(typeChecker.typeToString(typeChecker.getTypeAtLocation(declaration))); // any

Some libraries are allowing transformation plugins access to the type checker seems very rickety to me... plugin authors are probably unknowingly relying on their transformation running first before other transformations. I guess this isn't a big deal in most situations though.

[nonara]

[Edited]

Absolutely right, @dsherret . The current design passes one root node (SourceFile) at a time into a transformer function chain, which means it would require a redesign.

I agree that it doesn't seem sensible to have TS create a Program instance each time a transformer is called, as I imagine that could get quite costly and many transformers do not use a TypeChecker. I assume that it was designed the way it is, in part, to maximize performance.

With that said, of the few transformer tutorials that are out there, most give examples of valid cases which use a TypeChecker. Also, I can confirm that many of the transformers being used by ts-patch / ttypescript also use the TypeChecker.

I just recently discovered the potential issue that @dsherret mentioned when I was digging into transformNodes(), which is how I landed here. As maintainer of ts-patch, the potential issue does concern me; moreso, though, as one who writes transformers that sometimes use a TypeChecker, I can sympathize with the frustration.

I am currently writing something which uses ts.transform() to alter specific type declarations. It needs to extract information from the updated types after. In order to do this, I have to call ts.transform() twice.

I'd like to propose benign compromise to mitigate this which I believe will not affect any other operations or behaviour. I'd love if anyone on the TS team would weigh in.

Background:

Currently, transformNodes behaves in the following manner:

        // Chain together and initialize each transformer.
        const transformersWithContext = transformers.map(t => t(context));
        const transformation = (node: T): T => {
            for (const transform of transformersWithContext) {
                node = transform(node);
            }
            return node;
        };

        // Transform each node.
        const transformed = map(nodes, allowDtsFiles ? transformation : transformRoot);

The issue here is that it is transforming files on a per-file basis.

Proposal

What I suggest is the following:

• Modify the above code to transform on a per-transformer basis. (Each SourceFile in nodes is transformed before moving to the next transformer)
• Add something like getCurrentNodes() to the TransformerContext which would provide access to the current transformed state of nodes

These would be very small adjustments, but it would open up the possibility to use a TypeChecker with accurate representations of the current state of the AST.

How?

The tranformer can:

• Create a CompilerHost with customized getSourceFile that provides the nodes from context.getCurrentNodes()
• Create a new Program instance using the host, which would allow for an up-to-date TypeChecker

Synopsis

• There are many valid use-cases in which type-checking is useful during transform
• Current model causes that to be unreliable
• The proposed change is small and seems that it won't break, change, or slow down the current process.

Those things in mind, if the TS team agrees to have a look at an actual implementation, I'd be happy to submit a PR for review!