Julia: corruption when performing unsafe_load on a pointer to a type with a Ref field

[jrevels]

I was doing some fun unsafe experimenting when I noticed:

julia> immutable Foo
         x::Base.RefValue{Int}
       end

julia> x = [Foo(Ref(1))]
1-element Array{Foo,1}:
 Foo(Base.RefValue{Int64}(1))

julia> wat = unsafe_load(pointer(x))
Foo(Foo(Base.RefValue{Int64}(1)))

julia> dump(wat)
Foo
  x: Foo
    x: Base.RefValue{Int64}
      x: Int64 1

julia> versioninfo()
Julia Version 0.6.0-dev.80
Commit 2277270 (2016-08-08 16:56 UTC)
Platform Info:
  System: Darwin (x86_64-apple-darwin13.4.0)
  CPU: Intel(R) Core(TM) i5-4288U CPU @ 2.60GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, haswell)

cc @maleadt

[maleadt]

More corruption occurs if the Ref isn't the first field:

julia> versioninfo()
Julia Version 0.6.0-dev.112
Commit 008d8e7* (2016-08-09 16:45 UTC)
Platform Info:
  System: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.7.1 (ORCJIT, ivybridge)

julia> immutable Foobar
           foo::Int
           bar::Ref{Int}
       end

julia> i = 1;

julia> a = [Foobar(i, Ref(i))];

julia> a[1]
Foobar(1,Base.RefValue{Int64}(1))

julia> unsafe_load(pointer(a))
Foobar(140454668175984,#undef)

julia> unsafe_load(Base.unsafe_convert(Ptr{Foobar}, a))
Foobar(140454668175984,#undef)

[vtjnash]

I'm not quite sure what you expected when you loaded a random portion of a pointer array and told Julia to interpret it as a FooBar object

[vtjnash]

Also, if you had used Ref instead of pointer, I think this whole issue would have been avoided.

[jrevels]

Also, if you had used Ref instead of pointer, I think this whole issue would have been avoided.

Sure. Hence my preface:

I was doing some fun unsafe experimenting

It makes sense that unsafe_load will return nonsense if I try to interpret a pointer as a value, I guess I wasn't sure whether I should expect it to automatically perform the next "jump" down the chain. So if I want to pull this wacky stunt, I should be handling the extra indirection myself, then?

[JeffBezanson]

If your goal was just to have fun with unsafe operations, it seems you accomplished that :) Otherwise, it might help to say more about what you actually needed to do.

[jrevels]

Otherwise, it might help to say more about what you actually needed to do.

I'm implementing an unsafe immutable view into a specific field of the wrapped array's values. From a high level perspective, I'm trying to get a feel for the lower-bound on the number of allocations required for a benchmark of a reverse-mode AD prototype I'm working on.

Anyway, we can take this discussion offline if you're curious. I don't want to cause any more GitHub noise than I already have 😛