Julia: Behavior of Array[] with array elements changed in Julia v1.3.0-rc5 somehow

[oschulz]

With Julia 1.3.0-rc5 I get a lot of inference failures that weren't there before, when testing ArraysOfArrays:

(v1.3) pkg> add ArraysOfArrays
[...]

(v1.3) pkg> status ArraysOfArrays
    Status `/[...]/environments/v1.3/Project.toml`
  [65a8f2f4] ArraysOfArrays v0.4.2

(v1.3) pkg> test ArraysOfArrays
   Testing ArraysOfArrays
 Resolving package versions...
[...]
deepgetindex: Error During Test at /[...]/packages/ArraysOfArrays/ngj7Z/test/functions.jl:37
  Test threw exception
  Expression: #= /[...]/packages/ArraysOfArrays/ngj7Z/test/functions.jl:37 =# @inferred(deepgetindex(A, 1, 2)) === A[1, 2]
  return type Array{Int64,2} does not match inferred return type Array
[...]  
deepview: Error During Test at /[...]/packages/ArraysOfArrays/ngj7Z/test/functions.jl:72
  Test threw exception
  Expression: #= /[...]/packages/ArraysOfArrays/ngj7Z/test/functions.jl:72 =# @inferred(deepview(A, 1, 2, 2, 1)) == view(A[1, 2], 2, 1)
  return type SubArray{Int64,0,Array{Int64,2},Tuple{Int64,Int64},true} does not match inferred return type Any  
[...]
ERROR: LoadError: Some tests did not pass: 258 passed, 0 failed, 8 errored, 0 broken.
in expression starting at /[...]/packages/ArraysOfArrays/ngj7Z/test/runtests.jl:5
ERROR: Package ArraysOfArrays errored during testing

julia> versioninfo()
Julia Version 1.3.0-rc5.1
Commit 36c4eb251e (2019-11-17 19:04 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.1 (ORCJIT, skylake)
Environment:
  JULIA_PKG_DEVDIR = /[...]/dev
  JULIA_DEPOT_PATH = /[...]
  JULIA_PKGDIR = /[...]
  JULIA_NUM_THREADS = 6

With Julia v1.0 to v1.3.0-rc4 all is fine, no failed inferences.

Tested with official Julia binaries on Ubuntu 18.04 64-bit.

[oschulz]

@StefanKarpinski, you asked for a strong argument why this should be fixed before a v1.3 release. I don't know if the following counts, but at least it's a MWE without any package deps (except for Test):

using Test
A = hcat(Array[rand(2,3), rand(3,1)], Array[rand(2,2), rand(1,4)])
foo(A::AbstractArray{<:AbstractArray,N}, idxs::Vararg{Any,N}) where {N} = getindex(A, idxs...)
@inferred foo(A, 1, 1)

[oschulz]

Oh, it's not what I do with foo at all, it's just the creation of A.

Julia v1.0.5 to v1.3.0-rc4.1:

julia> typeof(hcat(Array[rand(2,3), rand(3,1)], Array[rand(2,2), rand(1,4)]))
Array{Array{Float64,2},2}

Julia v1.3.0-rc5.1

julia> typeof(hcat(Array[rand(2,3), rand(3,1)], Array[rand(2,2), rand(1,4)]))
Array{Array,2}

[oschulz]

So maybe it's just my tests that are written badly - could it be that that hcat(Array[ isn't even supposed to be type-stable?

[Liozou]

I don't think that is even related to hcat, simply your Array annotation seems to be confusing the compiler. Indeed, compare

julia> typeof(Array[rand(2,3)])
Array{Array,1}

julia> typeof(Array{Float64,2}[rand(2,3)])
Array{Array{Float64,2},1}

julia> typeof([rand(2,3)])
Array{Array{Float64,2},1}

As the last example shows, the compiler seems to be doing the work just fine, but the incomplete annotation changes the inference result (which I think is normal).

So if you remove the type annotation on your vectors you do get

julia> typeof(hcat([rand(2,3), rand(3,1)], [rand(2,2), rand(1,4)]))
Array{Array{Float64,2},2}

[KristofferC]

Probably https://github.com/JuliaLang/julia/commit/6eebbbe2d205f8116330a77ca5e15f4a356232db#diff-8cc03187983013adb308460f8365e1d0R1560 that changed this which indeed seems like it wouldn't do the tightening of the element type anymore?

[oschulz]

simply your Array annotation seems to be confusing the compiler.

Yes ... I had that in there for a reason, originally ... may have been necessary in pre-1.0 times, or so. Ooops.

Sorry!!

[JeffBezanson]

Oops, that one line change should be reverted. This is not intentional.

[oschulz]

Well, at least I didn't report something completely bogus, even though I massively overestimated the severity of the issue. That makes me feel a little better. :-)