Typescript: Feature Request: Macros

[Gaelan]

This one might be waaaaaay out of scope, but I think it is worth proposing. The idea here is to add support for macros, functions that run at compile time, taking one or more AST nodes and returning an AST node or array of AST nodes. Examples/use cases: (Syntax off the top of my head, open to better ideas)

Interface-Based Validation

// validation.macros.ts
function interfaceToValidatorCreator(interface: ts.InterfaceDeclaration): ts.Statement {
    // Would return something like "const [interface.name]Validator = new Validator({...});",
    // using types from the interface
}
macro CreateValidator(interface: ts.InterfaceDeclaration) {
    return [interface, interfaceToValidator(interface)];
}
// mainFile.ts
/// <macros path='./valididation.macros.ts' />
@#CreateValidator // Syntax 1: Decorator-Style
interface Person {
    name: string;
    age: number;
}

PersonValidator.validate(foo)

Type Providers

// swagger.macros.ts
macro GetSwaggerClient(url: ts.StringLiteral): AssertionExpression {
    // return something like "new SwaggerClient([url]) as SwaggerClientBase & {...}" where
    // ... is an object creating the methods generated from the URL.
}
// mainFile.ts
/// <macros path='./swagger.macros.ts' />
var fooClient = #GetSwaggerClient("http://foo.com/swagger.json"); // Syntax 2: Function call syntax
fooClient.getPeople((people) => {
    people.every((person) => console.log(person.firstName + "," + person.lastName);
});

Conditional Compilation

// conditional-compilation.macros.ts
macro IfFlagSet(flagName: ts.StringLiteral, code: ts.Statement[]): ts.Statement[] {
    return process.env[flagName.string] ? code : []
}
// mainFile.ts
/// <macros path='./compilation.macros.ts' />
#IfFlagSet("DEVELOPMENT") { // Syntax 3: Language Construct-Like (multiple arguments can be passed in parentheses)
    expensiveUnnecessarySanityCheck()
}

Notes

• Macros would run right after parsing. Not sure how we would deal with macros that need type information.
• This would make running tsc on unknown code as dangerous as running unknown code. It might be good to require a --unsafeAllowMacros argument, not settable from a tsconfig.json.
• It might be worth nothing in the docs that the AST format may change at any time, or something along those likes
• The macro keyword would probably compile to a function, followed by a ts.registerMacro(function, argumentTypes, returnType call.
• Macros must be typed as returning a AST interface. This means that functions creating ASTs will probably need to have an explicit return type (or a calling function could have an explicit return type.
  
  
  
  • Alternatively, we could consider giving the kind property special treatment in macros.ts files.
  
  
• Just because a proposed syntax looks like a normal typescript construct doesn't mean it behaves like one. #Foo(interface Bar{}) is valid syntax, as long as there is a macro named Foo that takes an interface.
  
  
  
  • Exception: The Decorator syntax might need to be a bit more choosy (no decorating 1 + 1, but decorating Interfaces, interface items, functions, etc. should be fine.
  
  
• This issue is likely to be updated quite a bit. For a log of changes, see the gist

[mhegazy]

duplicate of #3136?

[Gaelan]

@mhegazy I don't think so. Support for macros as AST->AST functions would let us do anything you could do with a type provider (just return an AssertionExpression, see the second example in the issue text), but also conditional compilation (return the passed code if the condition is true, otherwise do nothing), as well as general boilerplate reduction.

[arkdemon]

+1

[fgalassi]

+1

[elbarzil]

+1 (I almost expected it to work with sweet.js.)

[ulfryk]

+1

[nisimjoseph]

+1

[joergwasmeier]

You can also look at the haxe macro system how they implemented it. http://haxe.org/manual/macro.html

[soycabanillas]

+1

[Zorgatone]

+1 :+1:

[carywreams]

Question
Could this be provided by a pre-compile hook; between slurping the _.ts_ file and depositing the _.js_ file?

Possible Benefits

• Allows for competing macro processors, eventually yielding a best-in-class ? or 500 of them...
• Natural requirement is production of valid TypeScript; but that's now on the user
• Allowed to evolve separately and distinctly along a rigorous path for macro definitions and implementations (ala regular expressions)

---

_Of course, I could be missing some fundamental concept of macros and compilers, rather than just attempting to break up the traditional view._

[lazdmx]

+1

[gauravk92]

+1

[Zorgatone]

I need this. My needed use-case right now would be to make sort-of an "inline-function", or similar to a C-preprocessor-like parametric macro. That code would be inlined to avoid the function call on the JavaScript output.

C Macro Example:

#define ADD(x, y) ((x) + (y))

C inline Example:

inline int ADD(int x, int y) {
    return x + y;
}

I'd write something similar in TypeScript (let's assume the keyword inline will work only with functions that can be inlined).

In TypeScript the inline approach would look like this:

inline function ADD(x: number, y: number) {
    return x + y;
}

UPDATE: Looks like a similar issue was already here #661

[AlexGalays]

I would enjoy some form of macros for sure. As great as typescript is, it's still crappy old JS underneath.
Something I would want to macro in would be proper if/else expressions.

[crazyquark]

👍

[zpdDG4gta8XKpMCd]

migrated from #11536 (which was closed in favor of this one)

situation:

• sometimes i wish i could generate a piece of code based on some existing one, for example a construction function for an given interface

typescript interface MyConfig { name: string; values: number[]; } function myConfigFrom(name: string, values: number[]) : MyConfig { return { name, values }; }

problem: currently my options are

• either write it by hands (tedious monotonous work)
• put together a homemade code generator and run it as a pre-build step (lot of maintenance, non-standard)

solution:

• allow AST rewrites via decorators

typescript @rewrite(addContructorFunction) interface MyConfig { name: string; values: number[]; } function addContructorFunction(node: ts.Node): ts.Node[] { return [node, toConstructorFunction(node as ts.InterfaceDeclaration)]; } function toConstructorFunction(node: ts.IntefaceDeclaration): ts.FunctionDeclaration { // fun stuff goes here }

[bcherny]

This would be huge. In something like Scala, macros are a way for the community to implement and test out new language features that are not yet (or will never be) supported by the core language.

After adding macro support, TS would have a large laboratory of potential features to draw on when implementing new ones, and could gauge support and feasibility of a feature before implementing it.

Features like pattern matching could first be implemented as macros, and then either moved into a standard macro lib, or into TS core if they are broadly useful and popular. This takes a burden off TS maintainers and authors, and gives the community freedom to experiment without forking the TS compiler.

[elibarzilay]

FWIW, I think that a more promising direction is for a macro facility to accommodate TS. The obvious example would be to extend sweet.js so it accepts the TS syntax, and expands into TS code. This way, TS doesn't need to know about macros at all.

This leads to something very similar to Typed Racket (for anyone who knows that), including the minor disadvantage of not being able to write macros that depend on types.

[bcherny]

@elibarzilay With that approach, would macros be typesafe? If the whole point of TS is to be a typesafe layer on top of JS, macros should ideally also be typesafe.

Again comparing to Scala macros, their safety is a huge win. Otherwise you end up shooting in the dark without IDE/compiler support until you get something that compiles.

[elibarzilay]

@bcherny: The macro code itself wouldn't be typed. But that's minor IMO (since at that level it's all ASTs in and out). (Compared to random scala macros that I've seen after a few seconds of grepping the web, you get only Expr with no type qualification.)

The code that macros _produce_ might not be well typed, but it still goes through the type checker which does verify that the result is safe.

[zozzz]

I think this is something similar to c/c++ perprocessor maybe, with type check? But i really want to write something like that:

#IfFlagSet("DEVELOPMENT") {
    macro assert(cond: any, message?: string) {
          if (!cond) { throw new Error("...") }
    }
} else {
   macro assert(...x: any[])  // or something similar, and in this case dont emit code for this macro call
}

[elibarzilay]

(Similar, but a _proper_ macro system compared to CPP is like comparing JS to machine code...)

[TobiasHeidingsfeld]

+1

[wiltonlazary]

Hygienic macro. https://en.wikipedia.org/wiki/Hygienic_macro

[AlexGalays]

JS is a mixed bag. It has some nice features, and some really awful ones. Also, new features take a very long time to get voted, approved by TC39 then implemented by the browsers. Political agendas may sometimes block some great features.

Macros could help us implement some very useful things in user land. I would love to use this right now: https://github.com/mindeavor/es-pipeline-operator

[Coderah]

+1

My example is one of having interfaces (that also use extends) that describe websocket network messages with binary specific types (type int8 = number), and wanting to generate code for something like https://github.com/phretaddin/schemapack

Interestingly I've done exactly this without macros and it was a pain in the rear both for me and future developers to the project. But it also saved an incredible amount of time considering there were over 100 different network message interfaces.

[AlexGalays]

Another use case: Given a simple or discriminated union, generate the list of all the possible values. This can help with mistakes where things are modified in one place but not the other.

[fis-cz]

+1

[winksaville]

+1

[tycho01]

Reposting relevant new comment from @disnet over at sweet:

@vegansk various background tasks have been completed but nothing directly on supporting TS/Flow in sweet.
I'm definitely motivated since everything I write now is in flow (even sweet core!).
My current focus is updating our internal AST to the latest version of Shift (so we can support async/await). Once that has been handled we will be in a good position to support types.
If anyone wants to help out a good place to start is getting TS/Flow support added to Shift. We depend on shift codegen to render our AST so it will need to be extended to handle types.

[calebharris]

+1

[tycho01]

The request 'Plugin Support for Custom Transformers' (#14419) sounds potentially relevant for this.

Edit: oh, TS's exposed program.emit already allows passing custom transformers, but these only cover existing nodes, while macros allow adding new nodes.

[tycho01]

The Sweet thread has it that one issue the TS compiler cannot currently handle well itself is hygiene. I wonder if anyone from the team could comment on that.

Edit: related quote from declareSymbol:

If we do get an existing symbol, see if it conflicts with the new symbol we're creating. For example, a 'var' symbol and a 'class' symbol will conflict within the same symbol table. If we have a conflict, report the issue on each declaration we have for this symbol, and then create a new symbol for this declaration.

That sounds like over here it leaves picking new names the responsibility of the user. I'd imagine the issue may be encountered / dealt with in JS transpilation if it creates its own variables there...

Edit: oh, createTempVariable.

[stevenarnold]

This would truly be a killer feature for TypeScript and IMO might be a game-changer in the front-end language wars. As others have pointed out, macros would engage the community in evolving the language and providing highly expressive functionality to developers. In my case, I'd love to have a great pattern-matching syntax to go with TypeScript's union types. (I am not impressed much by typeof and instanceof and long if-then chains.) I think a good macro system could provide this, and I imagine once TypeScript users had a reliable and appealing pattern-matching syntax, adoption would be swift. From a general language maintenance point of view, this would give language maintainers a standard way to implement many new features in the language without changing the source code of the language. The focus would shift to ensuring the macro system was bug-free, while keeping the core stable with few changes. This should improve the overall stability of the language while still providing most of the desired features people need.

[bioball]

👍 👍 👍 This would be seriously awesome.

Chiming in here on the original syntax: I think there are two issues:

1. If the macro is meant to return a different type than the object being decorated, it gets pretty messy to figure out how to structure your code. In the validator example, where did PersonValidator come from?
2. macros as decorators don't work so well if I'm say, writing a validator to an interface that's defined in a 3rd party d.ts file. For example...

import { IFooThing } from "somelib";

// since I didn't define `IFooThing`, i'd have to delcare another interface or something?
@#CreateValidator
interface Bar extends IFooThing {}

Rather, I think macros would work better if the syntax worked like so:

const _createValidator = (node: ts.Node) => { ... };
const createValidator<T> = macro _createValidator;

const fooThingValidator = createValidator<IFooThing>;

fooThingValidator.validate(...)

[jedwards1211]

@Gaelan note that Babel users have solved this problem with Babel plugins instead of some kind of macro language embedded in the core compiler and your own code. Hence there is babel-plugin-flow-runtime for converting flow type declarations into runtime validators.

(and when you're talking about converting any type declaration to a validator, you're talking about something waaaaaaay more complex than what's typically considered a "macro").

For instance @zozzz there is a babel-plugin-transform-define that can replace process.env.NODE_ENV with 'production' (or whatever you configure it to use) and even remove if statements where if (process.env.NODE_ENV !== 'production') get transformed into if ('production' !== 'production').

[bcherny]

@jedwards1211 How would typechecking work? Babel gives syntax+grammar support, but it doesn’t typecheck, which is most of what TSC does. Having Scala-style macros built into ore would mean the compiler can verify (1) that the macro code typechecks, and (2) that the code it generates will typecheck.

[Coderah]

@jedwards1211 plugins to babel serve a great use, but like TSC they are for preprocessing, the clarity for a developer is low. @bcherny has done a great job of noting another huge reason to have them part of the language. Not to mention familiarity, writing a plugin requires a lot of knowledge of how Babel and Babel plugins works, instead of a fairly straightforward built-in syntax.

[elibarzilay]

Macros are compiler plugins, only specified within the language instead of externally. When they're specified within the language, rather than rewrite rules like CPP macros, you can do anything in them. For example, there is a Racket library that implements a macro that will fetch the metadata for a google service and produce glue code for it, all as part of the compiler's work (which is when macros get expanded).

[jedwards1211]

Can there really be a straightforward syntax for AST transforms? That's what Babel plugins do, and it's hard to imagine it being super straightforward.

Which personally I think is a good thing, because i think C/C++ macros have been heavily abused because they're so simple. You sometimes see projects with highly idiosyncratic ways of defining
types and functions.

Whereas with the level of investment it takes to write a Babel plugin, there's less temptation to write a one-off Babel plugin that will only ever be used in a single idiosyncratic project or file.

[elibarzilay]

Practical systems do the usual patterns + templates to match input and construct new syntax out of it. https://www.sweetjs.org/ does it for JS. The result of that is language that is almost as straightforward as rewrite rules, with holes that contain compile-time code that does more than plain rewrites.

[artem-v-shamsutdinov]

Right now I'm stuck doing this:

` // FIXME: Containers - Uncomment for production build
// @ContentChildren(forwardRef(() => MarginComponent))
// marginContainers: QueryList;
// @ContentChildren(forwardRef(() => RowComponent))
// rowContainers: QueryList;
// @ContentChildren(forwardRef(() => StepperComponent))
// stepperContainers: QueryList;

// FIXME: Containers - Comment out for production build
@ContentChildren(forwardRef(() => require('./margin/margin.component').MarginComponent))
marginContainers: QueryList;
@ContentChildren(forwardRef(() => require('./row/row.component').RowComponent))
rowContainers: QueryList;
@ContentChildren(forwardRef(() => require('./stepper/stepper.component').StepperComponent))
stepperContainers: QueryList;`

Having conditional compilation would help a lot!

+1

[Machiaweliczny]

Having macros would be nice 👍 but I think usage should be very explicit. I think it might be worth checking how they are done in Elixir as big part of Elixir language is written as macros - https://elixir-lang.org/getting-started/meta/macros.html

[xialvjun]

babel has macro long time ago: https://github.com/kentcdodds/babel-plugin-macros

[trsh]

This is must be

[marcelfalliere]

Wouldn't https://github.com/nippur72/ifdef-loader do the job here ? If you are using webpack of course.

[ulfryk]

:+1:

This would be an awesome feature…

[Frizi]

Macros would be a very useful feature not only as a future language features playground, but also as a way to add logging or debugging to existing codebases. Some informations about the code are lost during the compilation or are inaccessible at runtime (think types or variable names). Macros would allow pulling such things out of the code without manual work of putting that metadata into string literals. Also any kind of DSLs would be possible (think JSX) without explicit compiler support, which could help reduce the core size.

As far as type-checking goes, I like the Rust way of handling that. Macros are operating not on AST level, but on the token trees. Those token trees are type-checked themselves to some extend (you can't mix an identifier with a statement for example). Once the token tree is expanded by macro, it can be fully parsed to AST (which might generate an error) and by type-checked (again with potential error). All errors generated strictly inside macros are reported at the macro invocation. Any errors that come from tokens passed to the macro as an argument are reported at those tokens. That means if you have an type error on an expression passed to the macro, it will be highlighted properly.

[wojciechmorawski]

+1

[GuiAmPm]

+1

[pietrovismara]

Any news on this?

This issue is going on for 3 years now and I can't find any acknowledgement by contributors, like if the idea is even being considered or not.

It would be great to know if there's any intention to include this proposal in a roadmap or if it's just going to be dismissed, or even if there are further requirements to be developed in order for this feature to be implemented.

[nahidakbar]

+1
would make things a shit load easier to debug
e.g. __file__ can be the ts file
__line__ can be the ts line
__function__ can be the ts function name etc

[hoegge]

Yes, this would be really useful - and is really a big lack at the moment - , especially for trying out things, the language does not yet support - e.g. I need it for experimenting with seamless switching between async and sync code

[xiaoxiangmoe]

see https://github.com/xiaoxiangmoe/typescript-macros

[tycho01]

@xiaoxiangmoe that looks interesting! I was wondering, how does your approach compares to like sweet.js?

[xiaoxiangmoe]

@tycho01 This is not another implementation of sweetjs, but another implementation of babel-plugin-macros

[xiaoxiangmoe]

@tycho01 see https://github.com/kentcdodds/babel-plugin-macros/issues/94#issuecomment-447566994

I'm planing to provide a real macro in the future, but I don't know how to design it now.

[thinwire]

Let me add another bump to this. Support for inline functions and the ability to do conditional compilation is extremely useful in game development and realtime graphics work. I'm currently forced to use a bunch of sed commands to strip out dev-time stuff from the generated code, and I'd rather not do that.

[thinwire]

Another feature that this type of macro support could add would be (finally) mangled privates. We already have source maps; full names of private fields should not end up in the generated javascript and they certainly shouldn't cause field name clashes further up an inheritance chain!

[hktonylee]

+1 for macro support. This can eliminate the need of the utility classes for typing support (e.g. redux actions).

[danielpza]

Checkout https://github.com/LeDDGroup/typescript-transform-macros

[elibarzilay]

babel ≠ macro system

babel plugins are not macros.

[danielpza]

https://github.com/codemix/babel-plugin-macros is about macros though

[pietrovismara]

Checkout https://github.com/LeDDGroup/typescript-transform-macros

@danielpa9708
The example in the repo looks great, I would be extremely interested to achieve zero-cost abstractions (like shown in the example for array.map) but for classes, something like what happens with structs in rust.

Is something like this possible already?

[danielpza]

@pietrovismara I'm not acquainted with the structs in rust, could you elaborate more?

[KSXGitHub]

@danielpa9708 Rust macro is hygienic: You can use any "variable" names within a macro without worrying that it will capture actual identifiers from non-macro context.

Rust Macro

[elibarzilay]

@KSXGitHub, it's important to remember that hygiene has another side -- that you can use any identifiers in a macro without worrying that they will be captured by identifiers from the use context. (And IIRC, Rust indeed does that.)

@pietrovismara, using macros to avoid the cost of a proper function is generally a bad idea, and it's not the intended use for macros.

@danielpa9708, indeed that thing is about macros, but what it defines seems like a very weak start. (Mostly judging by the todo list, since I couldn't get it to work.) Also, it's true that babel could be used to implement macros, but it would be easier to go the other way: having proper macros would make many uses babel unnecessary. (Which would make things better, since it could focus on compilation rather than try to be a poor man's macro-system-like-thing.)

[KSXGitHub]

using macros to avoid the cost of a proper function is generally a bad idea

How is this a bad idea? It is true that most of the time you need to wrap repetitive code inside a function avoid duplication and reduce code size. But sometimes, functions are just helpers that contain no actual commands within them (e.g. compose, pipe, partial, x => x, etc.). Using an actual runtime function for these would be a waste.

and it's not the intended use for macros.

I agree, this is an intended use of const function (the likes of const fn in Rust and constexpr in C++).

Macros' indented use is metaprogramming.

[elibarzilay]

A complete explanation would be very off-topic, but trying to be super terse: (a) compilers are generally much better at doing inlining; (b) inlining functions is much easier than macros (since they have uniform semantics); (c) it's extremely easy to make mistakes. As an example of the last one, see the macro example that @danielpa9708 pointed to: const input = inputConst; is a subtle point that people can miss, and indeed that same example doesn't do that for visitor -- which means that if you use it with a visitor argument that is not a simple value (e.g., the result of a higher-order function), then that function would be called in each iteration of the loop.

(And BTW, inlining is not the same as constant folding, which is what those constexpr things. A good macro system can obviously make it easy to do similar things to both, but again, that's not the main reason to having one...)

[pietrovismara]

@elbarzil

I know that compilers are better at inlining, but in JS we have the problem that instantiating a class from some data has a high cost, especially when you deal with large amounts of data.

@danielpa9708

In Rust, structs are kinda like classes in JS, you can define properties and methods on them, but there is no cost in instantiating them since the compiler takes care of it. So you get the nice abstraction of classes, but with no runtime cost (hence zero-cost abstraction).

Looking at JS, there definitely is a cost in instantiating classes and depending on your use case (e.g game dev) it can be a significant factor forcing you to use other languages.

[danielpza]

https://github.com/LeDDGroup/typescript-transform-macros/commit/2268d05

feat: hygienic macros

[beenotung]

I've just made a small tool to write macro in Typescript and expand into Typescript. It's string-based, not AST-based though.
https://github.com/beenotung/tsc-macro

[beenotung]

You may also find this cli tool / library helpful, It can be used together with tsc-macro to generate Typescript type declaration from json data
https://github.com/beenotung/gen-ts-type

[fullofcaffeine]

@beenotung Do you know if there's anything out there that could help convert babel macro over to something like your ts-macro project? Or maybe you have some insights? I'd love to use https://github.com/ts-delight/if-expr.macro, but for my backend, I don't use babel, but tsc.

[beenotung]

@fullofcaffeine from my understanding, you may need to configure the build routine to pipe babel and tsc.

When I was using Angular 1, they didn't have official support on typescript, but I could write gulp task to first compile from typescript, then compile from angular compiler (then bundle with webpack or any further operation)

[beenotung]

Another way to 'get macro feature' with typescript is to use typedraft. It is a superset of typescript, with macro supported.
repo: https://github.com/mistlog/typedraft

[blainehansen]

I just barely built a syntactic macro capable compiler wrapper for typescript:
https://github.com/blainehansen/macro-ts

And an accompanying blog post talking about the why/how:
https://blainehansen.me/post/macro-ts/

It's pretty hacky, but if you're aching for true meta-programming in typescript, it gets the job done.

[K4rakara]

:+1: I'd love to see this in TypeScript, however I would say that the addition of a rust-esq procedural macro would also be nice:

Rust:

fn foo() -> Html {
    html! {
        <div class="foo">
           { "Hello world!" }
        </div>
    }
}
fn bar(v: &str) -> SqlQuery {
    sql! {
        SELECT *
        FROM users
        WHERE foo = @{v}
    }
}

TypeScript:

function foo(): Html {
    return #html {
        <div class="foo">
            { "Hello world!" }
        </div>
    }
}

function bar(v: string): SqlQuery {
    return #sql {
        SELECT *
        FROM users
        WHERE foo = @v
    }
}

[jedwards1211]

About generating validators from type defs...
This is super important, it's a holy grail of TS/Flow that they really ought to have builtin support for, but OP's example is clunky.
There are ways to not force any naming convention, and it should work on any common type, not just interfaces.

The syntax babel-plugin-flow-runtime supports, which works for Flow code, is close to perfect (though the plugin is clunky in other ways):

import {reify, type Type} from 'flow-runtime'

type Person = {name: string, age?: number}
const PersonType = (reify: Type<Person>) // babel plugin magic

PersonType.assert({name: 'dude', age: 50})

type Other = number
const OtherType = (reify: Type<Other>)

OtherType.assert(2)