Runtime: Make System.Collections.Immutable [Serializable]

[psmolinsky]

Please make collections from namespace System.Collections.Immutable serializable if it is possible - mark them with [Serializable] attribute and do other related changes. There can be many cases when this would be useful. Thanks.

[jaredpar]

Binary serialization isn't available in the portable subset so [Serializable] isn't possible. At least to my knowledge.

[weshaggard]

Correct @jaredpar the Serializable attribute isn't available in .NET Core because binary serialization isn't supported. @psmolinsky which serializer are you trying to use with Immutable collections?

[ellismg]

Tagging as "needs more info" to understand what serialization technologies @psmolinsky is using. As noted above, we don't have binary serialization in .NET Core.

[psmolinsky]

In my use case I'm trying to save some list of serializable objects to AppFabric cache. Serialization of generic List<> or array is fine. As far as immutable collections are mostly wrappers (ImmutableArray<> just wraps regular array), I don't understand why they also can't support serialization (be marked as [Serializable]). Thanks for clarification.

[stevozilik]

+1
trying to store ImmutableList in CallContext.LogicalSetData, for which it needs to be [Serializable]

[shmao]

[Serializable] attribute is not available in .Net Core. We discussed the possiblity of adding DataContract Surrogate in .Net Core (see https://github.com/dotnet/apireviews/pull/16) and @khdang is currently working on the feature(https://github.com/dotnet/corefx/issues/1931). When the surrogate type is available in .Net Core, it would be up to the owner of the System.Collections.Immutable Namespace to decide whether to support serialization of Immutable types using surrogates.

Assigning the issue back to @ellismg and removing the serializaiton label.

[Priya91]

SerializableAttribute is now available in .NET Core. @psmolinsky Can you come up with the PR for these changes and a proposal to be approved in api-review.

[danmosemsft]

@psmolinsky do you have interest in writing up the quick formal proposal here? (API is trivial, but with motivations)

See API review process

[safern]

Marking as up for grabs to see if there is some interest in doing the formal API proposal, if there is none we will just close it.

[mellinoe]

I think our current stance is that we don't want to add [Serializable] to any of our types if we don't absolutely have to, because of the huge compatibility burden it adds. @terrajobst , I think you have strong opinions about this?

[mellinoe]

Also adding @nguerrera , @AArnott

[AArnott]

I agree: it's preferable to keep serialization in the business of a serialization library rather than offering it in the type being serialized. Newtonsoft.Json and another popular super-compact-and-efficient serialization library (whose name escapes me at the moment) both support immutable collections natively.
For anyone currently asking for this, why do they need the support in the framework as opposed to using a 3rd party serialization library?

[fabricefou]

I agree with you it's better to use NewtonSoft.Json or Protobuf. And for new project that's what I'll do.
But in my team we have a lot of legacy binary serialization, so actually we can't use easily immutable collection in our legacy structures.
Make immutable collection serializable will help developers to use it in legacy project.

[sab39]

I'm in the same boat as @fabricefou - in my case a Web Forms app that stores a lot of custom objects in ViewState. Migrating many of these objects to use immutable instead of mutable collections should be a no-brainer, but the lack of serializability makes it impossible without complicated workarounds.

[bp74]

I've implemented the ISerializationSurrogateProvider interface for the most common System.Collection.Immutable types. It works pretty good with DataContractSerializer now.
Please take a look here: https://github.com/bp74/System.Collections.Immutable.Surrogate

[sharwell]

Please make sure that whatever serialization logic gets implemented (if any) does not produce results that expose the internal structure of these collections. Several of the collections currently use AVL trees, and we don't want serialization concerns to block progress, e.g. dotnet/corefx#1502.

Exposing the internals of these to serialization would be roughly equivalent to marking them [Obsolete]. We'd end up with a slow, legacy set of collection instances and a new set using more efficient structures.

[bp74]

@sharwell It is serialized like a standard List or Dictionary. If you look at the XML you won't see anything from the internal structure.

[aelij]

@bp74 I just published a very similar library. Main difference is that you don't have to configure the serializer (which sometimes isn't possible). And it would work with any serializer that supports standard .NET collections.

[Mardoxx]

[Serializable] Immutable* would be great.

At the moment I've got the following dirty workaround, e.g. with ImmutableDictionary<,>

[Serializable]
public class MyClass
{
    [NonSerialized]
    private IImmutableDictionary<string, int> dictionary;

    private readonly IDictionary<string, int> backingDictionary; // This is what gets serialized

    public MyClass()
    {
        backingDictionary = new Dictionary<string, int>(/* whatever */);
    }

    public IImmutableDictionary<string, int> Dictionary => dictionary ?? (dictionary = backingDictionary.ToImmutableDictionary());
}

This works by having an internal state which is [Serializable], and then lazy creates the Immutable. Works when you don't have any control over the IFormatter to notify it of your ISerializationSurrogates.

Kinda sucks though, because it's storing two collections (so ~2x number of pointers I'd guess?), but it's "good enough for me"

A better way would be to do your own manual serialization, then, for example, set the backing field to null so it can be GC'd. However if you are targeting netstandard2.0, and someone wants to use your lib in netframework, you can't because [OnDeserialized] etc... attributes don't work yet (https://github.com/dotnet/standard/issues/300)

[fabricefou]

Any news on this issue?

[fabricefou]

Some hope to see news on this subject?
@terrajobst when did you choose if you will support serialization for immutable collection?
Thanks

[fabricefou]

How to have more some visibility on this issue ?

[eiriktsarpalis]

Triage: would support such an addition, please provide an API proposal.

[fabricefou]

Triage: would support such an addition, please provide an API proposal

I don't understand, just add [Serializable] attribute is not an option ?

[safern]

Adding @ViktorHofer as the serialization expert in case he has something to consider before adding this.

[ViktorHofer]

We aren't planning to expand the support matrix of serializable types for .NET 5 onwards. There are concerns as support binary serialization (which uses the Serializable attribute) limits the underneath implementation because of implementation details being encoded in the serialization payload.

cc @stephentoub

[stephentoub]

I don't understand, just add [Serializable] attribute is not an option ?

No. Just doing that exposes the implementation details of the collection, and makes it effectively impossible for us to make any meaningful changes to the implementation. It's one of (but far from the only) reason we've significantly limited what types are serializable in .NET Core; most of the types that were serializable in .NET Framework are not in .NET Core, by design, with only a handful of hand-picked types opted-in as a compatibility tradeoff to help existing code migrate.

Binary serialization is not a technology we believe in, and in fact, there are active conversations like https://github.com/dotnet/runtime/issues/29976 about deinvesting in it further.

As noted by other respondents earlier in the thread, there are better technologies available for this that you can use around the collections you want to serialize rather than requiring changes to the collections themselves. And if you must serialize them with BinaryFormatter, you can use serialization surrogates to do so.

Given all that, we don't plan to make these collection types serializable.

Thanks.

[fabricefou]

Thanks @stephentoub for this answer.
I understand well your motivation to not incite people to use BinaryFormater.

Juste to mitigate some of your arguments :

• List<T> is serializable why not ImmutableArray<T> ? internal structure are very similar.
• on legacy system, BinaryFormater are often wildly use, don't add serializable attribute on immutable collection slow down it's use.
• surrogate are pretty hard to use on imbricated data structure ex: ImmutableArray>. BTW if you have smart implementation for immutableCollection don't hesitate to share.
• to not expose implementation details we can implement the ISerializable/IDeserializationCallback interfaces as used in Dictionary<K,V>

[sab39]

@stephentoub (sorry for the very belated response!)

And if you must serialize them with BinaryFormatter, you can use serialization surrogates to do so.

For those of us stuck in the legacy Framework world and stuck with binary serialization (ASP.NET WebForms forces both of those things on me), is there a way to put in some global code to automatically use serialization surrogates any time the formatter encounters one of the immutable collection classes? Or do I have to put in specific serialization handling every time I want to have a collection in a [Serializable] class?

[stephentoub]

@sab39, I'm not aware of any way to globally affect all BinaryFormatter instances. I believe you'd need to specify the surrogate on each instance.
cc: @GrabYourPitchforks

[GrabYourPitchforks]

@stephentoub is right, it's a per-instance thing. Were you also looking for global fallback surrogate selector support in Full Framework apps as well, or just Core?

[sab39]

@GrabYourPitchforks My particular situation is a Framework application using ASP.NET Web Forms. The Web Forms framework does a lot of implicit binary serialization for things like the ViewState object and my particular application is heavily dependent on putting lots of custom objects into ViewState. (Needless to say, those are questionable-at-best design decisions from both my app and from WebForms, which is probably why WebForms isn't being ported to Core, but I'm stuck with them)

My interest in this topic is that I'd love to be able to use Immutable collections in those custom objects rather than relying on List<> and Dictionary<>, but it's impractical to add a whole lot of serialization boilerplate to every class that happens to have a field that's a collection type.

I think WebForms does have a way to customize the Formatter that's used for its serialization, so it wouldn't have to be global to the entire process or AppDomain or anything like that. Just need something so the class _being_ serialized doesn't need to do anything beyond the usual [Serializable] to get its ImmutableCollection typed fields serialized and deserialized correctly.