Crystal: [RFC] Macro Defs - Methods in Macro land

[Blacksmoke16]

As someone who does a good amount with macros, one thing I find lacking is the ability to be DRY. There currently isn't a way (AFAIK) that allows encapsulating logic to be reused within different macros. I.e. if there is some piece of logic you need to do multiple times in a macro, you would have to duplicate it.

It would be a great addition to allow defining methods that can be used within macro code; I.e. that accept one or more ASTNodes, and return an ASTNode.

For example:

macro def foo(x : StringLiteral) : StringLiteral
  "foo#{x}"
end

macro bar
  {{ foo "x" }}
end

bar # => "foox"

This would allow common/complex logic to be defined once and reused throughout an application.

[asterite]

This is basically extending the macro language to allow user-defined macro methods. Right now they are hardcoded in the compiler and not extensible. Right?

My idea is that they are also macros. For example, you could reopen Crystal::StringLiteral to add methods to it:

class Crystal::StringLiteral
  macro foo(x)
    "foo#{x}"
  end
end

Then call it:

macro bar
  {{ "x".foo }}
end

It only makes sense to call the macro method foo inside macro code because that returns ASTNode, not code, though maybe we can allow it outside macro code and the code is then transformed to code by calling to_s on it.

So the way I'd like to see this implemented is by just allowing to find macros when you call them inside macro code.

That said, I'd like to avoid so many macro code in Crystal, and not having this is a great way to accomplish this.

[Blacksmoke16]

This is basically extending the macro language to allow user-defined macro methods.

Not necessarily. I would rather just having methods within macro land. I.e. you pass it one or more ASTNode, and it returns an ASTNode.

Like your example would be

macro def foo(x : StringLiteral) : StringLiteral
  "foo#{x}"
end

macro bar
  {{ foo "x" }}
end

bar # => "foox"

I don't really think it's necessary to be able to reopen the ASTNodes to add your own method like you can with the stdlib types. Using what is currently defined is usually good enough. All I'm suggesting is being able to have some sort of way to keep things DRY. As in this latest example, if you had to format x three times you would have to do "foo#{x}" three times as well.

EDIT: I updated the issue desc to reflect this.

[watzon]

As someone that's also been doing a lot with macros, especially lately, I love this idea. I actually needed something exactly like this last night and was out of luck. I also love @asterite's idea of being able to open up macro definitions and add methods to them like you could with any other class or struct. Would this all be insanely hard to do?

Additionally having initializers for ASTNode types would be nice too. Currently the only way to define a HashLiteral, ArrayLiteral, or anything else afaik is to actually use the literal notation. That means that using a macro to generate a Hash or Array can get kinda hacky.

[asterite]

Well, not crazy hard, I managed to implement a quick prototype in an hour.

After thinking a bit more about this, I think it would be really nice to go forward with this.

One thing I dislike about macros is how long and hard to understand they can get. That's a good way to try to avoid them. But if we can DRY them up and make them more concise and easier to understand, and more powerful, maybe it's the best solution.

[watzon]

100% agree. Crystal macros are powerful, but there are definitely some instances where you have to do some pretty hacky/unreadable things to do what you need to do. Documentation is also sparse, but once macros are somewhat finalized for v1 @Blacksmoke16 and I can probably work on that.

I feel like having initializers for ASTNode types would go a long way towards cleaning things up, as well as adding some methods that are missing on their standard library counterparts. Also, annotations need some work, but that's another issue.

[asterite]

I feel like having initializers for ASTNode types

Do you have an example of that? I don't quite understand what this means.

[Blacksmoke16]

{% hash = HashLiteral.new %}
versus
{% hash = {} of Nil => Nil %}

[watzon]

Exactly. Especially if we had stuff like ArrayLiteral.build and HashLiteral.new which takes a block and allows you to build it. As of right now, building a Hash using a macro is kinda hacky.

{% begin %}
{
  {% for c in SomeEnum.constants %}
    {{ c.id }} => {{c.stringify}}.camelcase
  {% end %}
}
{% end %}

With the ability to actually initialize types, especially using builder type methods, we could do something like this:

{% myhash = HashLiteral.build(SomeEnum.constants.size) do |hash, i| %}
  {% c = SomeEnum.constants[i] %}
  {% hash[c.id.stringify] = c.id.stringify %}
{% end %}
{{ myhash }}

Or something. Hopefully you get the idea. I did just realize that Hash doesn't actually have a method like this though. Maybe it should?

[asterite]

You can already build hashes with "regular" crystal code:

enum SomeEnum
  Red
  Green
  Blue
end

{% begin %}
  {% h = {} of Nil => Nil %}
  {% for c in SomeEnum.constants %}
    {% h[c.stringify] = c.stringify.downcase %}
  {% end %}
  p({{ h }})
{% end %}

So you see I create a hash and add elements to it at compile-time, instead of outputting a hash literal.

And with macro methods this would be simpler because you could define a method to do that, though you'd need to have each.

I don't think we'll add build and such methods, it's just too much in my opinion. We can't keep adding built-in macro methods.