Julia: provide a line continuation syntax for nicer macro calls

[musm]

currently

@inline @inbounds @another @one @for @fun function foo(x)
    x = x + 1
    x += 1
    return x + 1
end

Proposal: introduce a line continuation syntax, e.g. #\ (like how C uses \ or we could borrow from C and straight up use \, however # is nice since it already signifies a comment so it's more familiar) so that the following works

proposal

@inline @inbounds @another @one @for @fun #\
function foo(x)
    x = x + 1
    x += 1
    return x + 1
end

Motivation: it becomes hard and ugly (personal opinion) to identify functions when prefixed with multiple macros. Arguably the fact I'm defining a function should be front and center. In the current syntax one has to follow the chain of macro calls before being able to reach and identify the function signature. This proposal places the function signature front and center and allows the macro calls to placed above the signature.

More examples

currently

function foo(x,y)
    a = exp(1.3) 

    @simd @inbounds for i = 1:10
        x[i] = i + 2 + a*sin(1.0)
        y[i] = i + y[i]
    end
    x *= a
    return x, y
end

proposal

function foo(x,y)
    a = exp(1.3) 

    @simd @inbounds #\
    for i = 1:10
        x[i] = i + 2 + a*sin(1.0)
        y[i] = i + y[i]
    end
    x *= a
    return x, y
end

Again here it's hard to quickly identify the for loop due to the macro calls that precede the for loop definition. In the proposed syntax it's no more difficult than if no macros were used and one can quickly identify the for loop.

_The ultimate goal here is to keep the syntax simple and pretty, even if one has to use several macro calls_

[ararslan]

Currently you can just put the code in a begin/end block, or use the () syntax for macros. For example,

@simd @inbounds begin
    for i = 1:10
        x[i] = i + 2 + a*sin(1.0)
        y[i] = i + y[i]
    end
end

or

@simd @inbounds(
for i = 1:10
    x[i] = i + 2 + a*sin(1.0)
    y[i] = i + y[i]
end)

IMO both of these are quite readable, or at least not less so than #\.

[musm]

Right I'd argue the first is ugly and verbose the second is passable
....another ugly way as @TotalVerb mentioned is:

@simd @inbounds @othermac #=
=#for i = 1:10
    x[i] = i + 2 + a*sin(1.0)
    y[i] = i + y[i]
end)

Don't get hung up on #\ that was merely a suggestion to gauge feedback on the idea.

E.g. using \

@inline @inbounds @another @one @for @fun \
function foo(x)
    x = x + 1
    x += 1
    return x + 1
end

@simd @inbounds @othermac \
for i = 1:10
    x[i] = i + 2 + a*sin(1.0)
    y[i] = i + y[i]
end

In any case, if this feels unnecessary by any of the higher ups feel free to close. The strongest argument I can find against this is that at most, code blocks shouldn't need more than 3 macros and so readability shouldn't be that impacted that a new syntax is needed.

[DNF2]

Purely out of curiosity: What would happen if there was no line continuation syntax at all, that the macros were simply allowed to discard the leading newline and process the next valid expression? (Asking totally from a position of ignorance here.)

[yurivish]

@okvs I like that idea a lot -- it would allow for using macros in Python-like decorator style to annotate the block it belongs to -- but I think it introduces an ambiguity right now when a macro is defined with no arguments.

I wonder how often macros without arguments are used. If the rules were changed to require () for calls with no arguments, I think that would make this feature possible without any additional syntax.

[DNF2]

So the problem would be macros that take zero arguments. If macros could somehow tolerate being fed too many inputs, that could work, I guess, but I _really_ don't know what I'm talking about here...

I would just have liked to be able to do something like this:

@simd @inbounds
    for i = 1:10
        x[i] = i + 2 + a*sin(1.0)
        y[i] = i + y[i]
    end

[ararslan]

@okvs That would mean that macros that take no arguments would need to be called as @something().

[SalchiPapa]

@ararslan

Currently you can just put the code in a begin/end block, or use the () syntax for macros.

No you can't do that (at least not always):

julia> @simd @inbounds(
       for i = 1:10
           x[i] = i + 2 + a*sin(1.0)
           y[i] = i + y[i]
       end)
Base.SimdLoop.SimdError("for loop expected")

julia> @simd @inbounds begin
           for i = 1:10
               x[i] = i + 2 + a*sin(1.0)
               y[i] = i + y[i]
           end
       end
Base.SimdLoop.SimdError("for loop expected")

Also what happens with multiple arguments? You are only taking into consideration one arg.

[ararslan]

The @simd @inbounds was a bad example because it doesn't work anyway, no matter how you specify it.

julia> @simd @inbounds for i = 1:10
           println(i)
       end
ERROR: Base.SimdLoop.SimdError("for loop expected")

[TotalVerb]

Indeed, although begin creates a new :block expression, wrapping in parentheses is identical to the parentheses-less variant when it goes to the macro.

[musm]

reopening, the resolution/decision of https://github.com/JuliaLang/julia/pull/29273 should determine how this issue is closed.

[c42f]

I think it's fairly clear from #29273 that the specific implementation proposed there won't be accepted. But I still regularly want this feature and after some experience using the alternative of "commenting out the linebreak" I still find that ugly and distracting. To copy/summarize some thinking from https://github.com/JuliaLang/julia/pull/29273#issuecomment-429226473:

I feel this whole thing might still fly if we could come up with a compelling enough pair of ascii chars for line continuation.

Some desirable properties:

• It should be short but not steal useful character combinations. A character pair would be ideal here.
• Any text after a line continuation and on the same line should be considered a comment. So ideally the line continuation pair would end with the existing comment character #.
• It should be a syntax error in current julia so that non backward compatible code gives a clear syntax error in older versions and doesn't mean something entirely different.

There's not many viable character pairs which fit these criteria. .# is one option:

@inline @inbounds @another @one @for @fun  .#
function foo(x)
    x = x + 1
    x += 1
    return x + 1
end

[jekbradbury]

If a macrocall always consumed a newline if present (even when it turns out that macrocall takes zero arguments), would that introduce an ambiguity or break existing code?

[c42f]

always consumed a newline if present

I'm confused by this comment. Macro calls currently use the newline to terminate the argument list. That's a pretty critical feature of the syntax!

[jekbradbury]

D'oh, sorry, I see what the problem is now.

[pazner]

I often run into this issue (and the related issue #27533).

I think the proposals suggested in https://github.com/JuliaLang/julia/pull/29273#issuecomment-429226473 are quite good.

I took the liberty of implementing this in Julia's parser using the character combinations \# and .#. If people are interested, I can open a PR (or the existing PR #29273 can be modified to support this syntax). This could also be used to allow for where clauses on a new line (#31378).

What are the opinions on this?

[c42f]

Well I still think I had some good points about the design tradeoffs in https://github.com/JuliaLang/julia/pull/29273#issuecomment-429226473 but it didn't generate much discussion :man_shrugging: I'm certainly happy to continue the discussion here.

Generally I think it would be helpful to open a new PR for \# because having a candidate implementation is a great way to stimulate discussion (as you can see, https://github.com/JuliaLang/julia/pull/29273 generated useful ideas even though it's unlikely to be merged). If you implement \# we can try running PkgEval against it to see whether it does in fact break any packages. If it's non-breaking in practice we could consider it a minor change which can make it into 1.x.

Implementation details... I think \# should always lexed consistently as a line continuation token regardless of parser context (so not like what I did here: https://github.com/JuliaLang/julia/pull/29273/files#r218999435). There are some other design decisions to make. At minimum we need to decide whether \# is allowed based on whether it's currently parsing a macro... or we could just allow it in all cases. For a PR you could just allow it in all cases and see how that goes. (Another option is to go with the most conservative language change which is to only allow it while parsing macros; we can always relax that restriction later in a non-breaking way.) I'm not quite sure line continuation is the best way to allow where to continue a line — if that is doable without an explicit line continuation it might be better.

[c42f]

Another almost non-breaking option would be :# because in most circumstances having a : at the end of a line is already forbidden to help people coming from python get clear errors. (The exception is ternary syntax a ? b : c where the : is allowed at the end of line and could be succeeded by a #.)

Edit: actually there's some other cases where it's allowed. Eg, a = :

[tkf]

Was @# considered before? It may not be a nice syntax outside macros, though.

---

Edit: So, it'd be breaking because this works:

julia> @#
       inline f(x) = x
f (generic function with 1 method)

But do people use it? It is parsed like this

julia> """
       @#
       a
       """ |> Meta.parse |> dump
Expr
  head: Symbol macrocall
  args: Array{Any}((2,))
    1: Symbol @a
    2: LineNumberNode
      line: Int64 2
      file: Symbol none

[tkf]

So, it looks like you can implement line continuation as a macro

macro c(x)
    esc(x)
end

@#
c @inline @#
c @inbounds @#
c f(x, i) = x[i]

[c42f]

I remember thinking about @# though I'm not sure it was discussed.

I don't think anyone would be worried if @# stopped meaning what it currently does :laughing:

@ a is forbidden and yet the following is allowed:

@ #
a

[tkf]

@c is my favorite macro now :grinning:. I can make Julia look like Fortran.

OK, but thinking a bit more seriously, I think @# cannot be used since some code might be parsed as a valid Julia code with current parser:

@inbounds @#
f(x)[i] = 1

I think a new syntax should just throw an error in old parser as much as possible. I think it's too easy to get a currently valid code with @#.

[c42f]

To get the full beauty of Fortran shouldn't you have that c in the 6th column :-) ?

It's true that having both backward and forward compatibility would be nice. It wouldn't be great if the older parsers would happily interpret a new line continuation syntax as valid syntax meaning something completely different. It's true @# is very prone to this and \# seems somewhat the same. .# seems better from that point of view and :# might also be fine.

The implementation would be the same in any case.

[pazner]

@c42f I think your suggestions are good. I made a PR (#35336) for \#, we can continue to discuss there.