Julia: `inferred` macro causes :block Expr arguments to lose LineNumberNodes

[NHDaly]

I'm not sure if this is related to https://github.com/JuliaLang/julia/issues/20620 or not.

The @inferred macro causes block quotes in the arguments to a function to lose their LineNumberNodes:

julia> using Test

julia> f(e::Expr) = e
f (generic function with 1 method)

julia> f(quote 5+5 end)
quote
    #= none:1 =#
    5 + 5
end

julia> @inferred f(quote 5+5 end)
quote
    5 + 5
end

I'm assuming it's got something to do with the way the temporary args variable is generated, since it's already gone by that line:

julia> @macroexpand @inferred f(quote 5+5 end)
quote
    let
        begin
            #1378#args = ($(Expr(:copyast, :($(QuoteNode(quote
    5 + 5
end))))),)
            #1379#result = f(#1378#args...)
            #1380#inftypes = (Test.Base).return_types(f, (Test.Base).typesof(#1378#args...))
        end
        if (Test.length)(#1380#inftypes) == 1
            nothing
        else
            (Base.throw)((Base.AssertionError)("length(inftypes) == 1"))
        end
        #1381#rettype = if #1379#result isa Test.Type
                (Test.Type){#1379#result}
            else
                (Test.typeof)(#1379#result)
            end
        #1381#rettype == #1380#inftypes[1] || (Test.error)("return type $(#1381#rettype) does not match inferred return type $(#1380#inftypes[1])")
        #1379#result
    end
end

but i'm not sure what about that is wrong.

[NHDaly]

The relevant section of the @inferred macro is here:
https://github.com/JuliaLang/julia/blob/5fac1b547899ab4411c8de58654140ee4447dcc2/stdlib/Test/src/Test.jl#L1383-L1387

[NHDaly]

Oh, haha, jk, i guess it's probably this call to Base.remove_linenums! that's doing it:
https://github.com/JuliaLang/julia/blob/5fac1b547899ab4411c8de58654140ee4447dcc2/stdlib/Test/src/Test.jl#L1369

What's the reasoning behind that call? Can we do it in a way that it doesn't affect the arguments to the expression?

[KristofferC]

https://github.com/JuliaLang/julia/commit/3bf50dfd6921d389b3bd8b0caefcd87f26f1767e

[NHDaly]

😆 Haha thanks. yeah it took me a while to notice that call there, literally spelling out my complaint. 😝

remove line numbers from inferred test macro
avoids interfering with line numbers in containing file

The reasoning there makes sense. Any idea about how to restructure this to avoid removing them from user input? I'm playing with it now, but Expr traversals man...

[vtjnash]

We could just drop the call. The issue it was working around is fixed. (or with more difficultly, change the structure the macro to avoid calling it on the user input, :(let args = ($([esc(ex.args[i]) for i = 2:length(ex.args)]...),); $(Base.remove_linenums!(quote ... end))); end))

[NHDaly]

:(let args = ($([esc(ex.args[i]) for i = 2:length(ex.args)]...),); $(Base.remove_linenums!(quote ... end))); end)

Ah, damn, that makes sense. It's better than what i came up with, which is pretty silly in retrospect:
https://github.com/JuliaLang/julia/pull/31335/files

[NHDaly]

I was finally picking this up again, and i noticed that actually @test does the same thing, which was added here:
https://github.com/JuliaLang/julia/commit/639621859863609c5f3abbc2ed75c675695b3693#diff-bef5fc2625b92f0ae308eb13927125abR185

julia> e = quote 2 + 2 end
quote
    #= none:1 =#
    2 + 2
end

julia> @test quote 2+2 end == e
Test Failed at none:1
  Expression: $(Expr(:quote, quote
    2 + 2
end)) == e
   Evaluated: begin
        2 + 2
    end == begin
        #= none:1 =#
        2 + 2
    end
ERROR: There was an error during testing

Can I make the same change to @test? That is, to remove the linenumbers from the test code, but not from Expr arguments?

[NHDaly]

We could just drop the call. The issue it was working around is fixed. (or with more difficultly, change the structure the macro to avoid calling it on the user input, :(let args = ($([esc(ex.args[i]) for i = 2:length(ex.args)]...),); $(Base.remove_linenums!(quote ... end))); end))

And actually, unfortunately, @vtjnash i don't think the solution you presented would work here; that still leaves linenumbers, as far as I can tell. I think the problem is that the let ... end block ends up inserting line-numbers:

julia> :(let args = (1,2,3); $(Base.remove_linenums!(quote 2+2 end)); end)
:(let args = (1, 2, 3)
      #= REPL[5]:1 =#
      begin
          2 + 2
      end
  end)

[NHDaly]

After thinking more about this, I wonder if the right solution is maybe to change Base.remove_linenums! to not recurse into esc() Expr trees.

I think this would solve the above problem, and match the intended behavior of Base.remove_linenums!:

• The reason we use Base.remove_linenums! is to avoid injecting the linenumbers of the Test code into the user's test expressions. So we want to remove linenums from any code _iff_ it was introduced by _us_ and not the user.
• Conversely, we want to always esc _any code provided by the user_ exactly once, and we _don't_ want to esc() our code, since we want our variable names to get gensym'd.

So I think that 1:1 correspondence is a nice property that we can take advantage of to solve this problem, by just changing Base.remove_linenums! to not recurse into Expr(:escape, ...) trees. Does that seem reasonable to you? :)

---

EDIT: I have implemented this remove_linenums! solution, here: https://github.com/JuliaLang/julia/pull/31335