Julia: "Error showing value of type: `type TypeVar has no field var`" - error displaying MethodInstance encountered from `MathOptInterface`

[NHDaly]

When I was using the mechanism introduced in #37749 to log all of the MethodInstances that are inferred during execution of some code, I've encountered a MethodInstance that throws an error when it is displayed:

Error showing value of type Core.Compiler.Timings.Timing:
ERROR: type TypeVar has no field var
Stacktrace:
   [1] getproperty(x::TypeVar, f::Symbol)
     @ Base ./Base.jl:33
   [2] show(io::IOContext{IOBuffer}, x::Type)
     @ Base ./show.jl:799
   [3] sprint(f::Function, args::Type; context::IOContext{Base.TTY}, sizehint::Int64)
     @ Base ./strings/io.jl:103

As you can see, the MethodInstance and its specTypes do not print via the normal show():

julia> gt.mi_info.mi
MethodInstance for operate(::typeof(+), ::Type{T}, ::MathOptInterface.ScalarAffineFunction{T}, ::Error showing value of type Core.MethodInstance:
ERROR: type TypeVar has no field var
Stacktrace:


julia> gt.mi_info.mi.specTypes
Tuple{typeof(MathOptInterface.Utilities.operate), typeof(+), Type{T}, MathOptInterface.ScalarAffineFunction{T}, Error showing value of type UnionAll:
ERROR: type TypeVar has no field var
Stacktrace:

So in order to get more information about it, I've used static_show() to print it:

julia> function static_shown(x)
           p = Pipe()
           Base.link_pipe!(p, reader_supports_async=true, writer_supports_async=true)
           ccall(:jl_static_show, Cvoid, (Ptr{Cvoid}, Any), p.in, x)
           @async close(p.in)
           return read(p.out, String)
       end

Here's the method instance that fails to print:

julia> static_shown(gt.mi_info.mi.specTypes)
"Tuple{typeof(MathOptInterface.Utilities.operate), typeof(Base.:(+)), Type{T}, MathOptInterface.ScalarAffineFunction{T}, T where T<:T} where T<:(Blobs.Blob{_A} where _A)"

julia> static_shown(gt.mi_info.mi)
"operate(typeof(Base.:(+)), Type{T<:(Blobs.Blob{_A} where _A)}, MathOptInterface.ScalarAffineFunction{T<:(Blobs.Blob{_A} where _A)}, T where T<:T<:(Blobs.Blob{_A} where _A)) where {T<:(Blobs.Blob{_A} where _A)} from operate(Union{typeof(Base.:(+)), typeof(Base.:(-))}, Type{T}, MathOptInterface.ScalarAffineFunction{T}, Union{MathOptInterface.SingleVariable, MathOptInterface.ScalarAffineFunction{T}, T}) where {T}"

One weird thing that stands out to me is that from what I can tell this is the EXACT method instance that seems to be causing a StackOverflow in julia 1.6 during type inference, as reported in https://github.com/JuliaLang/julia/issues/22787! So maybe these are related?

Another weird thing is that if I copy/paste the output from static_shown into the REPL, it actually displays correctly!:

julia> Tuple{typeof(MathOptInterface.Utilities.operate), typeof(Base.:(+)), Type{T}, MathOptInterface.ScalarAffineFunction{T}, T where T<:T} where T<:(Blobs.Blob{_A} where _A)
Tuple{typeof(MathOptInterface.Utilities.operate),typeof(+),Type{T},MathOptInterface.ScalarAffineFunction{T},T} where T<:(Blob{_A} where _A)

Which makes me think there's some other error in the construction of the type that's not being captured by static_show()...

[NHDaly]

Also weird, the body and var print okay on their own, they just don't print correctly _together_:

julia> gt.mi_info.mi.specTypes.body
Tuple{typeof(MathOptInterface.Utilities.operate), typeof(+), Type{T<:(Blobs.Blob{_A} where _A)}, MathOptInterface.ScalarAffineFunction{T<:(Blobs.Blob{_A} where _A)}, T where T<:T<:(Blobs.Blob{_A} where _A)}

julia> gt.mi_info.mi.specTypes.var
T<:(Blobs.Blob{_A} where _A)

julia> gt.mi_info.mi.specTypes
Tuple{typeof(MathOptInterface.Utilities.operate), typeof(+), Type{T}, MathOptInterface.ScalarAffineFunction{T}, Error showing value of type UnionAll:
ERROR: type TypeVar has no field var
Stacktrace:
  [1] getproperty(x::TypeVar, f::Symbol)
    @ Base ./Base.jl:33
  [2] show(io::IOContext{Base.TTY}, x::Type)
    @ Base ./show.jl:799
  [3] show_datatype(io::IOContext{Base.TTY}, x::DataType)
    @ Base ./show.jl:875

Also there are a _lot_ of seemingly redundant type variables in there..

[NHDaly]

It turns out this method is specialized on a bunch of different types and 16 / 88 of them don't print:

julia> const mi_problems = Any[]
Any[]

julia> for mi in MethodAnalysis.methodinstances(MathOptInterface.Utilities.operate)
           try
               repr(mi)
           catch e
               push!(mi_problems, mi)
               @show e
           end
       end
e = ErrorException("type TypeVar has no field var")
e = ErrorException("type TypeVar has no field var")
...

julia> mi_problems
16-element Array{Any,1}:
Error showing value of type Array{Any,1}:
ERROR: type TypeVar has no field var

Here are a few of them:

julia> static_shown(mi_problems[5])
"operate(typeof(Base.:(+)), Type{T<:Tuple{Base.OneTo{T} where T<:Integer}}, MathOptInterface.ScalarAffineFunction{T<:Tuple{Base.OneTo{T} where T<:Integer}}, T where T<:T<:Tuple{Base.OneTo{T} where T<:Integer}) where {T<:Tuple{Base.OneTo{T} where T<:Integer}}"

julia> static_shown(mi_problems[6])
"operate(typeof(Base.:(+)), Type{T<:(Blobs.Blob{_A} where _A)}, MathOptInterface.ScalarAffineFunction{T<:(Blobs.Blob{_A} where _A)}, T where T<:T<:(Blobs.Blob{_A} where _A)) where {T<:(Blobs.Blob{_A} where _A)}"

julia> static_shown(mi_problems[7])
"operate(typeof(Base.:(+)), Type{T<:(Ptr{_A} where _A)}, MathOptInterface.ScalarAffineFunction{T<:(Ptr{_A} where _A)}, T where T<:T<:(Ptr{_A} where _A)) where {T<:(Ptr{_A} where _A)}"

This last one (7) I think should be reproducible with only the MathOptInterface package.

[NHDaly]

In case this is helpful, here's an upload of the above mi_problems[7] Type Tuple serialized via Serialization:

bad-mi-operate-+-ScalarAffineFunction.jlserialized.zip

This was serialized on julia 1.5.2, like this:

julia> serialize("/tmp/bad-mi-operate-+-ScalarAffineFunction.jlserialized", mi_problems[7].specTypes)

Here's some version info:

  [b8f27783] MathOptInterface v0.9.17

julia> versioninfo()
Julia Version 1.5.2
Commit 5beddbf698 (2020-10-06 17:36 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin19.6.0)
  CPU: Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

[JeffBezanson]

Another weird thing is that if I copy/paste the output from static_shown into the REPL, it actually displays correctly!:

This can sometimes happen if there are name collisions in UnionAll variables, such that the output doesn't actually correspond to the same type.

This seems to be due to the non-normalized T where T<:T, causing the UnionAll to collapse during show. I wonder where it came from.

[NHDaly]

Another weird thing is that if I copy/paste the output from static_shown into the REPL, it actually displays correctly!:

This can sometimes happen if there are name collisions in UnionAll variables, such that the output doesn't actually correspond to the same type.

Yeah, makes sense. thanks!

This seems to be due to the non-normalized T where T<:T, causing the UnionAll to collapse during show. I wonder where it came from.

Hmm yeah, weird. I'm not sure, either. Do you think this is an issue that comes from MathOptInterface somehow? Or is it an issue in julia?

And also, do you think this is indeed related to https://github.com/JuliaLang/julia/issues/22787? It seems weird to me that this same method instance showed up in both issues.

[NHDaly]

I walked through with the Debugger on julia 1.5.3, and i think the results are interesting. I'll verify them on 1.6 once I have an example test case up and running on 1.6 again.

It seems like the problem occurs here, where the call to UnionAll _doesn't produce a UnionAll,_ and instead produces a DataType:
https://github.com/JuliaLang/julia/blob/788b2c77c10c2160f4794a4d4b6b81a95a90940c/base/show.jl#L532

So then the subsequent call to x.var, right below, throws an exception, because type DataType has no field var:
https://github.com/JuliaLang/julia/blob/788b2c77c10c2160f4794a4d4b6b81a95a90940c/base/show.jl#L539

---

I verified this locally by reconstructing the variables, and indeed, calling UnionAll returns a DataType. Is that a bug?:

julia> mi.specTypes
Tuple{typeof(MathOptInterface.Utilities.operate),typeof(+),Type{T},MathOptInterface.SingleVariable,Error showing value of type UnionAll:
ERROR: type DataType has no field var
[...]

julia> x = mi.specTypes.body.parameters[5]
T where T<:MathOptInterface.ScalarAffineFunction{T<:(MathOptInterface.ScalarAffineFunction{T} where T<:MathOptInterface.AbstractScalarFunction)}

julia> typeof(x)
UnionAll

julia> T1 = @eval Base let x=$x, io=$(IOContext(stdout, :unionall_env => mi.specTypes.var));
                   counter = 1
                   newname = Symbol(x.var.name, counter)
                   newtv = TypeVar(newname, x.var.lb, x.var.ub)
       end
T1<:MathOptInterface.ScalarAffineFunction{T<:(MathOptInterface.ScalarAffineFunction{T} where T<:MathOptInterface.AbstractScalarFunction)}

julia> x{T1}
T1<:MathOptInterface.ScalarAffineFunction{T<:(MathOptInterface.ScalarAffineFunction{T} where T<:MathOptInterface.AbstractScalarFunction)}

julia> UnionAll(T1, x{T1})
MathOptInterface.ScalarAffineFunction{T<:(MathOptInterface.ScalarAffineFunction{T} where T<:MathOptInterface.AbstractScalarFunction)}

julia> typeof(UnionAll(T1, x{T1}))
DataType

I guess this is the problem. Is this a bug in UnionAll? Or is the entire type tuple malformed somehow? Or is it simply a bug in show() that we could fix?

It would be really great to fix this, since it currently throws an exception when attempting to build a FlameGraph from the result of @snoopi_deep, when it attempts to print these method instances.

Thanks! :)

[NHDaly]

Here are the results on 1.6 -- it seems to behave similarly incorrectly, except now it produces a TypeVar instead of a UnionAll, which still seems incorrect?:

julia> mi.specTypes
Tuple{typeof(MathOptInterface.Utilities.operate), typeof(+), Type{T}, MathOptInterface.ScalarAffineFunction{T}, Error showing value of type UnionAll:
ERROR: type TypeVar has no field var
Stacktrace:
  [1] getproperty(x::TypeVar, f::Symbol)
    @ Base ./Base.jl:33
  [2] show(io::IOContext{Base.TTY}, x::Type)
    @ Base ./show.jl:801
  [3] show_datatype(io::IOContext{Base.TTY}, x::DataType)
    @ Base ./show.jl:877
  [4] show(io::IOContext{Base.TTY}, x::Type)
    @ Base ./show.jl:775
  [5] show(io::IOContext{Base.TTY}, x::Type)
    @ Base ./show.jl:801
  [6] show(io::IOContext{Base.TTY}, #unused#::MIME{Symbol("text/plain")}, x::Type)
    @ Base ./show.jl:750
[...]

julia> x = mi.specTypes.body.parameters[5]
T where T<:T<:(Blobs.Blob{Delve.PagedDataStructures.BeMessageValue{V}} where V)

julia> typeof(x)
UnionAll

julia> T1 = @eval Base let x=$x, io=$(IOContext(stdout, :unionall_env => mi.specTypes.var));
                   counter = 1
                   newname = Symbol(x.var.name, counter)
                   newtv = TypeVar(newname, x.var.lb, x.var.ub)
       end
T1<:T<:(Blobs.Blob{Delve.PagedDataStructures.BeMessageValue{V}} where V)

julia> x{T1}
T1<:T<:(Blobs.Blob{Delve.PagedDataStructures.BeMessageValue{V}} where V)

julia> UnionAll(T1, x{T1})
T<:(Blobs.Blob{Delve.PagedDataStructures.BeMessageValue{V}} where V)

julia> typeof(UnionAll(T1, x{T1}))
TypeVar