Julia: Unintentional serialize breakage between 0.5/0.6?

[quinnj]

I'm seeing incompatibilities between serialize/deserialize between 0.5/0.6. Repro:

Julia 0.5:

julia> d = Date(2017, 3, 28)
2017-03-28

julia> f = open("test_deserialize", "w")
IOStream(<file test_deserialize>)

julia> serialize(f, d)

julia> close(f)

then on 0.6:

julia> d = deserialize(open("test_deserialize"))
ERROR: UndefVarError: Dates not defined
Stacktrace:
 [1] deserialize(::SerializationState{IOStream}, ::Type{Module}) at ./serialize.jl:610
 [2] handle_deserialize(::SerializationState{IOStream}, ::Int32) at ./serialize.jl:589
 [3] deserialize(::SerializationState{IOStream}) at ./serialize.jl:549
 [4] deserialize_datatype(::SerializationState{IOStream}) at ./serialize.jl:837
 [5] handle_deserialize(::SerializationState{IOStream}, ::Int32) at ./serialize.jl:579
 [6] deserialize(::IOStream) at ./serialize.jl:546

Then going the other direction:
0.6:

julia> f = open("test_deserialize", "w")
IOStream(<file test_deserialize>)

julia> serialize(f, Date(2017, 3, 28))

julia> close(f)

0.5:

julia> deserialize(open("test_deserialize"))
ERROR: TypeError: getfield: expected Symbol, got SimpleVector
 in deserialize(::SerializationState{IOStream}, ::Type{Module}) at ./serialize.jl:602
 in handle_deserialize(::SerializationState{IOStream}, ::Int32) at ./serialize.jl:581
 in deserialize(::SerializationState{IOStream}) at ./serialize.jl:541
 in deserialize_datatype(::SerializationState{IOStream}) at ./serialize.jl:822
 in handle_deserialize(::SerializationState{IOStream}, ::Int32) at ./serialize.jl:571
 in deserialize(::IOStream) at ./serialize.jl:538

Totally understand if this is intentional (I know we don't guarantee serialize compat between versions), but just wondered if this was intentional or something that could be fixed.

[quinnj]

Actually, thinking more about this, I can see this being due to the type system overhaul: internally, a Date object is:

julia> dump(d)
Date
  instant: Base.Dates.UTInstant{Base.Dates.Day}
    periods: Base.Dates.Day
      value: Int64 736416

so I wonder if the changes in parameterized types is yielding the change in serialization. If so, that's perfectly fine, we can close this and I'll find a way to deal. The error though, is a bit weird UndefVarError: Dates not defined.

[StefanKarpinski]

Serialization has never been intended to be stable across versions...

[JeffBezanson]

Yes, 0.6 changes the serialization format yet again. It might be possible to make a copy of the 0.5 serialize.jl, make a few tweaks, and load it on 0.6 for reading older files.

[quinnj]

That'd be nice, we could put it in a package. I'm realizing it might not be ideal to be relying on serialize to store values in a database. It's so handy though. I guess if we have a good "migration" solution, it's not too bad.

[malmaud]

Isn't this what JLD is for?

[StefanKarpinski]

The biggest issue with JLD is that some serialization situations are quite slow, but yes, this is what JLD is for.