Latest master, OSX
julia> sz = 3576
3576
julia> struct works
data::NTuple{sz, UInt8}
works() = new(ntuple(x->0x00, sz))
end
julia> @time works()
0.034114 seconds (27.93 k allocations: 1.803 MiB)
and
julia> const sz2 = 3576
3576
julia> struct doesntwork
data::NTuple{sz2, UInt8}
doesntwork() = new(ntuple(x->0x00, sz2))
end
julia> @time doesntwork()
Internal error: encountered unexpected error in runtime:
StackOverflowError()
intersect at /Users/jacobquinn/julia/src/subtype.c:1897
intersect_unionall_ at /Users/jacobquinn/julia/src/subtype.c:1510
intersect_unionall at /Users/jacobquinn/julia/src/subtype.c:1555
intersect at /Users/jacobquinn/julia/src/subtype.c:1914
intersect_ufirst at /Users/jacobquinn/julia/src/subtype.c:1189 [inlined]
intersect_var at /Users/jacobquinn/julia/src/subtype.c:1244
intersect_invariant at /Users/jacobquinn/julia/src/subtype.c:1759
intersect at /Users/jacobquinn/julia/src/subtype.c:1999
intersect_tuple at /Users/jacobquinn/julia/src/subtype.c:1657 [inlined]
intersect at /Users/jacobquinn/julia/src/subtype.c:1965
intersect_unionall_ at /Users/jacobquinn/julia/src/subtype.c:1510
intersect_unionall at /Users/jacobquinn/julia/src/subtype.c:1555
intersect at /Users/jacobquinn/julia/src/subtype.c:1914
intersect_all at /Users/jacobquinn/julia/src/subtype.c:2035
jl_type_intersection_env_s at /Users/jacobquinn/julia/src/subtype.c:2139
jl_typemap_intersection_node_visitor at /Users/jacobquinn/julia/src/typemap.c:481
jl_typemap_intersection_visitor at /Users/jacobquinn/julia/src/typemap.c:545
jl_typemap_intersection_visitor at /Users/jacobquinn/julia/src/typemap.c:536
ml_matches at /Users/jacobquinn/julia/src/gf.c:2381
jl_matching_methods at /Users/jacobquinn/julia/src/gf.c:1615
abstract_call_gf_by_type at ./inference.jl:1650
...
...
...
We need to fix inference to be non-recursive in order to handle all of the recursive functions in base defined on tuples (in this case, the one that gets it into trouble is code_typed(convert, (Type{NTuple{sz, UInt8}}, Tuple{Vararg{UInt8}}))
, where sz >≈ 1000). I think this is blocked on changing to a very linear IR however.
Both are broken now inside inference
Was this fixed? On my machine on latest master (a1da84c3b0406e8378d8af781fe74ce749725d1f) actually both of @quinnj's examples work for me! :)
However, i'm here because I _did_ see almost this exact same error newly show up during type inference for a part of our system that uses JuMP. It was succeeding on julia 1.5, but now fails with a StackOverflow inside inference as of a recent master.
Internal error: encountered unexpected error in runtime:
StackOverflowError()
intersect at /Users/nathandaly/src/julia/src/subtype.c:0
intersect_invariant at /Users/nathandaly/src/julia/src/subtype.c:2813
intersect at /Users/nathandaly/src/julia/src/subtype.c:3065
intersect at /Users/nathandaly/src/julia/src/subtype.c:0
intersect_all at /Users/nathandaly/src/julia/src/subtype.c:3110
intersect_var at /Users/nathandaly/src/julia/src/subtype.c:0
...
intersect at /Users/nathandaly/src/julia/src/subtype.c:0
intersect_unionall_ at /Users/nathandaly/src/julia/src/subtype.c:2565
intersect_unionall at /Users/nathandaly/src/julia/src/subtype.c:2616
intersect at /Users/nathandaly/src/julia/src/subtype.c:2989
intersect_all at /Users/nathandaly/src/julia/src/subtype.c:3134
jl_type_intersection_env_s at /Users/nathandaly/src/julia/src/subtype.c:3342
jl_type_intersection2 at /Users/nathandaly/src/julia/src/gf.c:1557
ml_matches at /Users/nathandaly/src/julia/src/gf.c:2874
jl_matching_methods at /Users/nathandaly/src/julia/src/gf.c:1839
_methods_by_ftype at ./reflection.jl:815 [inlined]
#findall#208 at ./compiler/methodtable.jl:55 [inlined]
findall##kw at ./compiler/methodtable.jl:52
jfptr_findallYY.YY.kw_9981 at /Users/nathandaly/src/julia/usr/lib/julia/sys.dylib (unknown line)
_jl_invoke at /Users/nathandaly/src/julia/src/gf.c:0 [inlined]
jl_apply_generic at /Users/nathandaly/src/julia/src/gf.c:2373
#210 at ./compiler/methodtable.jl:65
...
typeinf_local at ./compiler/abstractinterpretation.jl:1386
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1442
_typeinf at ./compiler/typeinfer.jl:205
typeinf at ./compiler/typeinfer.jl:200
typeinf_ext at ./compiler/typeinfer.jl:824
typeinf_ext_toplevel at ./compiler/typeinfer.jl:857
typeinf_ext_toplevel at ./compiler/typeinfer.jl:853
jfptr_typeinf_ext_toplevel_12121 at /Users/nathandaly/src/julia/usr/lib/julia/sys.dylib (unknown line)
_jl_invoke at /Users/nathandaly/src/julia/src/gf.c:0 [inlined]
jl_apply_generic at /Users/nathandaly/src/julia/src/gf.c:2373
jl_apply at /Users/nathandaly/src/julia/src/./julia.h:1690 [inlined]
jl_type_infer at /Users/nathandaly/src/julia/src/gf.c:300
jl_generate_fptr at /Users/nathandaly/src/julia/src/jitlayers.cpp:316
jl_compile_method_internal at /Users/nathandaly/src/julia/src/gf.c:1924
_jl_invoke at /Users/nathandaly/src/julia/src/gf.c:2183 [inlined]
jl_apply_generic at /Users/nathandaly/src/julia/src/gf.c:2373
attach_optimizer at /Users/nathandaly/.julia/packages/MathOptInterface/k7UUH/src/Utilities/cachingoptimizer.jl:150
optimize! at /Users/nathandaly/.julia/packages/MathOptInterface/k7UUH/src/Utilities/cachingoptimizer.jl:211
_jl_invoke at /Users/nathandaly/src/julia/src/gf.c:0 [inlined]
jl_apply_generic at /Users/nathandaly/src/julia/src/gf.c:2373
#optimize!#100 at /Users/nathandaly/.julia/packages/JuMP/e0Uc2/src/optimizer_interface.jl:130
optimize! at /Users/nathandaly/.julia/packages/JuMP/e0Uc2/src/optimizer_interface.jl:106 [inlined]
optimize! at /Users/nathandaly/.julia/packages/JuMP/e0Uc2/src/optimizer_interface.jl:106 [inlined]
I think c604df0f8d9d1c2eef4cba3910bee3dcd034d715 fixed the examples here. You'd probably need to poke at this in a debugger (same gdb below) to figure out what that case is (probably not quite the same as this).
thread 1
frame 10 # up or down
print x
print y
thread 2
p jl_(x)
p jl_(y)
Also helpful is doing negative backtraces:
bt -150
frame #####
Partial reduction of the case we've hit:
using JuMP
using Clp
using MathOptInterface
function fluff!(m::MathOptInterface.Utilities.CachingOptimizer)
MathOptInterface.Utilities.attach_optimizer(m)
# Without the following call, this segfaults.
# With the following call, this stack overflows.
# With or without this call, it's a party.
MOI.optimize!(m.optimizer)
end
model_backend = backend(Model(Clp.Optimizer))
fluff!(model_backend)
Interestingly optimization level appears to impact segfault versus stackoverflow behavior: The comments above apply at O1, but at O2 it's stackoverflows all around.
Further reduction
# Segfault repro at O1 / stackoverflow at O2
using JuMP
using Clp
using MathOptInterface
model = backend(Model(Clp.Optimizer))
MOI.copy_to(model.optimizer, model.model_cache, copy_names=false)
(Also, for clarification, the above stackoverflow we're seeing is _new_ as of julia 1.6, and wasn't occurring in 1.5.)
I just hit this bug like this i think (on 1.6 dev)
...
jfptr_typeinf_ext_toplevel_13560 at /home/ptah/julia/usr/lib/julia/sys.so (unknown line)
_jl_invoke at /home/ptah/julia/src/gf.c:2191 [inlined]
jl_apply_generic at /home/ptah/julia/src/gf.c:2373
jl_apply at /home/ptah/julia/src/julia.h:1690 [inlined]
jl_type_infer at /home/ptah/julia/src/gf.c:300
jl_generate_fptr at /home/ptah/julia/src/jitlayers.cpp:316
jl_compile_method_internal at /home/ptah/julia/src/gf.c:1924
jl_compile_method_internal at /home/ptah/julia/src/gf.c:1878 [inlined]
_jl_invoke at /home/ptah/julia/src/gf.c:2183 [inlined]
jl_apply_generic at /home/ptah/julia/src/gf.c:2373
attach_optimizer at /home/ptah/.julia/packages/MathOptInterface/k7UUH/src/Utilities/cachingoptimizer.jl:150
_jl_invoke at /home/ptah/julia/src/gf.c:2191 [inlined]
jl_apply_generic at /home/ptah/julia/src/gf.c:2373
#setup_model#2 at /home/ptah/.julia/dev/Hypatia/examples/common_JuMP.jl:76
unknown function (ip: 0x7f685e1fdbff)
_jl_invoke at /home/ptah/julia/src/gf.c:2191 [inlined]
jl_apply_generic at /home/ptah/julia/src/gf.c:2373
setup_model at /home/ptah/.julia/dev/Hypatia/examples/common_JuMP.jl:61
unknown function (ip: 0x7f685e1fb0d9)
_jl_invoke at /home/ptah/julia/src/gf.c:2191 [inlined]
...
and so on
The MRE in https://github.com/JuliaLang/julia/issues/22787#issuecomment-710701248 allowed me to bisect the stack overflow to 51592abc99d6ff0bf76c00202914cc0efd97b8e3, i.e. https://github.com/JuliaLang/julia/pull/37616. @vtjnash, any thoughts? :)
If it's helpful, I just ran @snoopi_deep
on the function that stackoverflows, and this leaves the internal timings
array in a state that shows the stack trace of methods it was in the middle of inferring when it threw the error. So we can go back and look at all the nodes currently in the stack to print out the stack trace of where it failed. I'll paste that here in case it's useful:
julia> timing = SnoopCompileCore.@snoopi_deep begin
@eval TPCH.run_query(conf, 17)
end
2020:10:20 22:24:59 [ tpch | info ] running Q17
Internal error: encountered unexpected error in runtime:
StackOverflowError()
intersect at /Users/nathandaly/src/julia/src/subtype.c:2965
intersect_all at /Users/nathandaly/src/julia/src/subtype.c:3110
intersect_var at /Users/nathandaly/src/julia/src/subtype.c:0
...
julia> show(stdout, MIME"text/plain"(), [node.mi_info for node in Core.Compiler.Timings._timings])
37-element Vector{Core.Compiler.Timings.InferenceFrameInfo}:
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for ROOT(), 0x0000000000000000, Any[], Any[])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for (::MathOptInterface.var"#copy_to##kw")(::NamedTuple{(:copy_names,), Tuple{Bool}}, ::typeof(MathOptInterface.copy_to), ::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}), 0x00000000000072ed, Any[], Any[NamedTuple{(:copy_names,), Tuple{Bool}}, Core.Const(MathOptInterface.copy_to), MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for #copy_to#3(::Base.Iterators.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:copy_names,), Tuple{Bool}}}, ::typeof(MathOptInterface.copy_to), ::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}), 0x00000000000072ed, Any[], Any[Base.Iterators.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:copy_names,), Tuple{Bool}}}, Core.Const(MathOptInterface.copy_to), MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for (::MathOptInterface.Utilities.var"#automatic_copy_to##kw")(::NamedTuple{(:copy_names,), Tuple{Bool}}, ::typeof(MathOptInterface.Utilities.automatic_copy_to), ::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}), 0x00000000000072ed, Any[], Any[NamedTuple{(:copy_names,), Tuple{Bool}}, Core.Const(MathOptInterface.Utilities.automatic_copy_to), MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for #automatic_copy_to#113(::Bool, ::Nothing, ::typeof(MathOptInterface.Utilities.automatic_copy_to), ::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}), 0x00000000000072ed, Any[], Any[Bool, Core.Const(nothing), Core.Const(MathOptInterface.Utilities.automatic_copy_to), MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for default_copy_to(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, ::Bool, ::Nothing), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, Bool, Core.Const(nothing), Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for try_constrain_variables_on_creation(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, ::MathOptInterface.Utilities.IndexMap, ::typeof(MathOptInterface.add_constrained_variables), ::typeof(MathOptInterface.add_constrained_variable)), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, MathOptInterface.Utilities.IndexMap, Core.Const(MathOptInterface.add_constrained_variables), Core.Const(MathOptInterface.add_constrained_variable), Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for copy_vector_of_variables(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, ::MathOptInterface.Utilities.IndexMap, ::Type{var"#s248"} where var"#s248"<:MathOptInterface.AbstractSet, ::typeof(MathOptInterface.add_constrained_variables)), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, MathOptInterface.Utilities.IndexMap, Type{var"#s248"} where var"#s248"<:MathOptInterface.AbstractSet, Core.Const(MathOptInterface.add_constrained_variables), Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_constrained_variables(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.AbstractVectorSet), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.AbstractVectorSet, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_variables(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::Any), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, Any, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_constrained_variables(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Reals), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_keys_for_bridge(::MathOptInterface.Bridges.Variable.Map, ::MathOptInterface.Bridges.var"#78#79"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, DataType}, ::MathOptInterface.Reals), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.Variable.Map, MathOptInterface.Bridges.var"#78#79"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, DataType}, MathOptInterface.Reals, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for call_in_context(::MathOptInterface.Bridges.Variable.Map, ::Int64, ::MathOptInterface.Bridges.var"#78#79"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, DataType}), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.Variable.Map, Int64, MathOptInterface.Bridges.var"#78#79"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, DataType}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for (::MathOptInterface.Bridges.var"#78#79"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, DataType})(), 0x00000000000072ed, Any[], Any[])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for bridge_constrained_variable(::Type{MathOptInterface.Bridges.Variable.FreeBridge{T}}, ::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Reals) where T, 0x00000000000072ed, Any[Any], Any[Type{MathOptInterface.Bridges.Variable.FreeBridge{T}} where T, MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_constrained_variables(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Nonnegatives), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Nonnegatives, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_keys_for_bridge(::MathOptInterface.Bridges.Variable.Map, ::MathOptInterface.Bridges.var"#78#79", ::MathOptInterface.Nonnegatives), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.Variable.Map, MathOptInterface.Bridges.var"#78#79", MathOptInterface.Nonnegatives, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for unbridged_map(::MathOptInterface.Bridges.Variable.SOCtoRSOCBridge{T}, ::Vector{MathOptInterface.VariableIndex}) where T, 0x00000000000072ed, Any[Any], Any[MathOptInterface.Bridges.Variable.SOCtoRSOCBridge, Vector{MathOptInterface.VariableIndex}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for rotate_unbridged_map(::Type, ::Vector{MathOptInterface.VariableIndex}, ::Vector{MathOptInterface.VariableIndex}), 0x00000000000072ed, Any[], Any[Type, Vector{MathOptInterface.VariableIndex}, Vector{MathOptInterface.VariableIndex}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for copy_single_variable(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, ::MathOptInterface.Utilities.IndexMap, ::Type{MathOptInterface.GreaterThan{Float64}}, ::typeof(MathOptInterface.add_constrained_variable)), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}, MathOptInterface.Utilities.IndexMap, Core.Const(MathOptInterface.GreaterThan{Float64}), Core.Const(MathOptInterface.add_constrained_variable), Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_constrained_variable(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.GreaterThan{Float64}), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.GreaterThan{Float64}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_key_for_bridge(::MathOptInterface.Bridges.Variable.Map, ::MathOptInterface.Bridges.var"#80#81"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.GreaterThan{Float64}, DataType}, ::MathOptInterface.GreaterThan{Float64}), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.Variable.Map, MathOptInterface.Bridges.var"#80#81"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.GreaterThan{Float64}, DataType}, MathOptInterface.GreaterThan{Float64}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for call_in_context(::MathOptInterface.Bridges.Variable.Map, ::Int64, ::MathOptInterface.Bridges.var"#80#81"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.GreaterThan{Float64}, DataType}), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.Variable.Map, Int64, MathOptInterface.Bridges.var"#80#81"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.GreaterThan{Float64}, DataType}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for (::MathOptInterface.Bridges.var"#80#81"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.GreaterThan{Float64}, DataType})(), 0x00000000000072ed, Any[], Any[])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for bridge_constrained_variable(::Type{MathOptInterface.Bridges.Variable.VectorizeBridge{Float64, S}}, ::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.GreaterThan{Float64}) where S, 0x00000000000072ed, Any[Core.Const(Float64), Any], Any[Type{MathOptInterface.Bridges.Variable.VectorizeBridge{Float64, S}} where S, MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.GreaterThan{Float64}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_constrained_variables(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::AbstractVector{var"#s32"} where var"#s32"<:MathOptInterface.AbstractScalarSet), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, AbstractVector{var"#s32"} where var"#s32"<:MathOptInterface.AbstractScalarSet, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_constrained_variable(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.AbstractScalarSet), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.AbstractScalarSet, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_variable(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_constrained_variables(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Reals), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_keys_for_bridge(::MathOptInterface.Bridges.Variable.Map, ::MathOptInterface.Bridges.var"#78#79"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, DataType}, ::MathOptInterface.Reals), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.Variable.Map, MathOptInterface.Bridges.var"#78#79"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, DataType}, MathOptInterface.Reals, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for call_in_context(::MathOptInterface.Bridges.Variable.Map, ::Int64, ::MathOptInterface.Bridges.var"#78#79"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, DataType}), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.Variable.Map, Int64, MathOptInterface.Bridges.var"#78#79"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, DataType}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for (::MathOptInterface.Bridges.var"#78#79"{MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, DataType})(), 0x00000000000072ed, Any[], Any[])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for bridge_constrained_variable(::Type{MathOptInterface.Bridges.Variable.FreeBridge{T}}, ::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Reals) where T, 0x00000000000072ed, Any[Any], Any[Type{MathOptInterface.Bridges.Variable.FreeBridge{T}} where T, MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Reals, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_constrained_variables(::MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, ::MathOptInterface.Nonnegatives), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.LazyBridgeOptimizer{MathOptInterface.Utilities.CachingOptimizer{Clp.Optimizer, MathOptInterface.Utilities.UniversalFallback{MathOptInterface.Utilities.Model{Float64}}}}, MathOptInterface.Nonnegatives, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for add_keys_for_bridge(::MathOptInterface.Bridges.Variable.Map, ::MathOptInterface.Bridges.var"#78#79", ::MathOptInterface.Nonnegatives), 0x00000000000072ed, Any[], Any[MathOptInterface.Bridges.Variable.Map, MathOptInterface.Bridges.var"#78#79", MathOptInterface.Nonnegatives, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for unbridged_map(::MathOptInterface.Bridges.Variable.SOCtoRSOCBridge{T}, ::Vector{MathOptInterface.VariableIndex}) where T, 0x00000000000072ed, Any[Any], Any[MathOptInterface.Bridges.Variable.SOCtoRSOCBridge, Vector{MathOptInterface.VariableIndex}])
Core.Compiler.Timings.InferenceFrameInfo(MethodInstance for rotate_unbridged_map(::Type, ::Vector{MathOptInterface.VariableIndex}, ::Vector{MathOptInterface.VariableIndex}), 0x00000000000072ed, Any[], Any[Type, Vector{MathOptInterface.VariableIndex}, Vector{MathOptInterface.VariableIndex}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}, Union{}])
julia>
(It's _probably_ not relevant, but the chunk containing the last 9 lines are repeated once earlier, so there's some recursion going on here as well.)
i think this is more or less just showing the same thing @Sacha0 reduced it to, now that i'm looking at it!
Further reduction
using JuMP
using Clp
using MathOptInterface
model = backend(Model(Clp.Optimizer))
function fluff(dest::MOI.ModelLike, src::MOI.ModelLike, cis_src)
cis_src = MOI.get(model.model_cache, MOI.ListOfConstraintIndices{MOI.VectorOfVariables, MathOptInterface.AbstractVectorSet}())
set = MOI.get(src, MOI.ConstraintSet(), first(cis_src))
MOI.add_constrained_variables(dest, set)
end
fluff(model.optimizer, model.model_cache, cis_src)
Or taking off from Nathan's snoopi_deep trace, we can more directly trigger what I would hope is the same stack overflow
using MathOptInterface
atypes = Tuple{Type, Vector{MathOptInterface.VariableIndex}, Vector{MathOptInterface.VariableIndex}}
meth = first(methods(MathOptInterface.Bridges.Variable.rotate_unbridged_map, atypes))
optimize = true
sparams = Core.svec()
types = Tuple{typeof(MathOptInterface.Bridges.Variable.rotate_unbridged_map), atypes.parameters...}
interpreter = Core.Compiler.NativeInterpreter(typemax(UInt))
Core.Compiler.typeinf_code(interpreter, meth, types, sparams, optimize)
yielding
ERROR: StackOverflowError:
Stacktrace:
[1] _methods_by_ftype
@ ./reflection.jl:821 [inlined]
[2] #findall#208
@ ./compiler/methodtable.jl:57 [inlined]
[3] (::Core.Compiler.var"#210#211"{Int64, Core.Compiler.CachedMethodTable{Core.Compiler.InternalMethodTable}, Core.Box})()
@ Core.Compiler ./compiler/methodtable.jl:67
[4] get!
@ ./iddict.jl:163 [inlined]
[5] #findall#209
@ ./compiler/methodtable.jl:66 [inlined]
[6] abstract_call_gf_by_type(interp::Core.Compiler.NativeInterpreter, f::Any, argtypes::Vector{Any}, atype::Any, sv::Core.Compiler.InferenceState, max_methods::Int64)
@ Core.Compiler ./compiler/abstractinterpretation.jl:71
[7] abstract_call_known(interp::Core.Compiler.NativeInterpreter, f::Any, fargs::Vector{Any}, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState, max_methods::Int64)
@ Core.Compiler ./compiler/abstractinterpretation.jl:1003
[8] abstract_call(interp::Core.Compiler.NativeInterpreter, fargs::Vector{Any}, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState, max_methods::Int64)
@ Core.Compiler ./compiler/abstractinterpretation.jl:1026
[9] abstract_call(interp::Core.Compiler.NativeInterpreter, fargs::Vector{Any}, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState)
@ Core.Compiler ./compiler/abstractinterpretation.jl:1010
[10] abstract_eval_statement(interp::Core.Compiler.NativeInterpreter, e::Any, vtypes::Vector{Any}, sv::Core.Compiler.InferenceState)
@ Core.Compiler ./compiler/abstractinterpretation.jl:1132
[11] typeinf_local(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
@ Core.Compiler ./compiler/abstractinterpretation.jl:1386
[12] typeinf_nocycle(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
@ Core.Compiler ./compiler/abstractinterpretation.jl:1442
[13] _typeinf(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
@ Core.Compiler ./compiler/typeinfer.jl:205
[14] typeinf(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
@ Core.Compiler ./compiler/typeinfer.jl:200
[15] typeinf_code(interp::Core.Compiler.NativeInterpreter, method::Method, atypes::Any, sparams::Core.SimpleVector, run_optimizer::Bool)
@ Core.Compiler ./compiler/typeinfer.jl:761
[16] top-level scope
@ REPL[8]:1
Reduces to
using MathOptInterface
atypes = Tuple{typeof(+), Type, MathOptInterface.ScalarAffineFunction, MathOptInterface.ScalarAffineFunction}
sigtype = Tuple{typeof(MathOptInterface.Utilities.operate), atypes.parameters...}
first(Core.Compiler._methods_by_ftype(sigtype, -1, typemax(UInt)))
where _methods_by_ftype
immediately ccall
s jl_matching_methods
.
Given how this affects JuMP, I'm requesting to make sure this is fixed for the 1.6 release.
A little more information. This call to jl_type_intersection2
appears to result in the stack overflow:
(lldb) c
Process 7713 resuming
Process 7713 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 6.1
frame #0: 0x000000010217b2c6 libjulia-debug.1.6.dylib`jl_type_intersection2(t1=0x00000001257157f0, t2=0x0000000125715bf0, isect=0x00007ffeefbfc7f0, isect2=0x00007ffeefbfc7a8) at gf.c:1555:6
1552
1553 static int jl_type_intersection2(jl_value_t *t1, jl_value_t *t2, jl_value_t **isect, jl_value_t **isect2)
1554 {
-> 1555 *isect2 = NULL;
1556 *isect = jl_type_intersection(t1, t2);
1557 if (*isect == jl_bottom_type)
1558 return 0;
Target 0: (julia-debug) stopped.
(lldb) p jl_(t1)
Tuple{typeof(MathOptInterface.Utilities.operate), typeof(Base.:(+)), Type{T}, T, MathOptInterface.ScalarAffineFunction{T}} where T<:(MathOptInterface.ScalarAffineFunction{T} where T)
(lldb) p jl_(t2)
Tuple{typeof(MathOptInterface.Utilities.operate), typeof(Base.:(+)), Union{Type{T}, Type{T}}, Union{MathOptInterface.ScalarAffineFunction{T}, MathOptInterface.ScalarAffineFunction{T}}, Union{MathOptInterface.ScalarAffineFunction{T}, T}} where T<:(MathOptInterface.ScalarAffineFunction{T} where T) where T
Reduced:
typeintersect(Tuple{Type{T}, T, Ref{T}} where T<:Ref,
Tuple{Type{S}, Union{Ref{S}, Ref{T}}, T} where T<:Ref where S)
@JeffBezanson do you have a handle on this? Is this branch blocking or just release blocking?
I would say just release blocking.
Most helpful comment
Given how this affects JuMP, I'm requesting to make sure this is fixed for the 1.6 release.