Crystal: Macro expressions ignore comments

[KimBurgess]

Lines which are commented appear to not be respected by the macro parser. As an example:

macro a(x)
  # {% if x %}
  puts 42
  # {% end %}
end

a true
a false

Result in the following output:

Note: single puts only.
https://play.crystal-lang.org/#/r/7x5y

[bew]

What would be the expected output?

[KimBurgess]

42

[bcardiff]

The macro evaluation in comments is desired to generate documentation.

https://github.com/crystal-lang/crystal/blob/774cc13fd1b8a5cdc4b0832f46ec3573c42f5e1c/spec/compiler/crystal/tools/expand_spec.cr#L596-L624

I'm not sure it will have a solution. If you are commenting as a temporal edit on the source, then we don't want a syntax error, but treat it as a raw comment. But on the other hand macros are expanded in comments.

Preventing for/if going out of a comment block or even preventing them in any comment will be confusing since some macro expressions will still work.

[asterite]

duplicate of #1318

[straight-shoota]

Theoretically, we could limit macro interpretation to only comments that are actually doc comments. The OP shows simply inline comments that are not used for doc.

This should be be possible to implement, although maybe not trivial. It might also be confusing when macros work in some comments but not in others.
I don't think this needs immediate action, but could be considered in the future.

[KimBurgess]

Thanks for the clarification, it makes perfect sense with that comment interpolation use case in mind.

[KimBurgess]

I've added a little section to the crystal-book with this info (see above). Not sure if this is the best place to discuss, but popping it here for future consideration. Could there be value in introducing syntax for a 'hard comment' to allow commenting out of macro directives and interpolation? Something similar to:

#! This is a line that will not be evaluated by the macro parser.
#! Doing this is ok: {{not_defined}}.

[straight-shoota]

Adding another comment syntax seems to be quite a huge cost for this. I don't think it's worth it.

But maybe we should really reconsider the macro semantics with comments.

The thing is in the example in the OP, the macro expansion doesn't make sense. The if-macro start and end end tags are in commented lines, the body is uncommented.
I can't think of any reason why you would want to have that behave any different way than what the OP intended: disable the macro conditional.

IMO it would be reasonable to force matching macro control tags to be either fully inside a single comment block or not commented at all.
This effectively scopes commented macro tags to a single comment block.

Also, macros can currently "escape" from the comment:

macro foo(bar)
  # {{ puts bar.id }}
end

foo "\n true" # => true

It doesn't seem like there would be a valid use case for theis either.

[jhass]

I don't know, trying to do some magic here feels wrong. You do some mistake and are surprised why a commented macro is still expanding or why a comment macro isn't. I prefer it to error in those cases rather than silently switch behavior because "smart".

Instead of a special comment syntax for macros we could think about a special syntax or rather method for macros to run in comments. Something analog to verbatim perhaps.

{% comment do %}
  # This is a comment whose {{macro}} expressions are expanded
{% end %}
# The {{macro}} expressions in this comment are not run anymore

[straight-shoota]

@jhass We don't need this to be invisible magic. It could just be a syntax error when a commented macro affects uncommented code. That's an easy enough rule and avoids surprises like the one described in the OP. When you think you commented a macro but in fact it doesn't.

Jumping on the idea of special comment syntax, if we want to explore this route, I'd not add a new keyword/macro method for this. But we could differentiate between normal comments indicated by # and special comments indicated by ##. This would be kind of similar to JavaDoc's docblock indicated by /** vs. a normal comment by /*.

I'm not convinced we need to differentiate comments though. A little "magic" to only enable macro expansion in a doc comment as described in https://github.com/crystal-lang/crystal/issues/8408#issuecomment-548384825 is IMO superior because it can just work out of the box, should meet almost all use cases and doesn't need any special syntax.