Runtime: SerializationException: Type 'System.Collections.Hashtable+SyncHashtable' in Assembly 'System.Runtime.Extensions, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' is not marked as serializable

[ershadnozari]

I have ported a number of class libraries from .NET Framework 4.5 to .NET Standard 2.0. Using these libraries from a .NET Framework 4.8 console application works fine. However referencing the libraries from a .NET Core 2.2 console app, results in the following exception:

SerializationException: Type 'System.Collections.Hashtable+SyncHashtable' in Assembly 'System.Runtime.Extensions, Version=4.2.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' is not marked as serializable

with stack trace

   at System.Runtime.Serialization.Formatters.Binary.ObjectReader.CheckSerializable(Type t)
   at System.Runtime.Serialization.Formatters.Binary.ObjectReader.ParseObject(ParseRecord pr)
   at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Parse(ParseRecord pr)
   at System.Runtime.Serialization.Formatters.Binary.BinaryParser.ReadObjectWithMapTyped(BinaryHeaderEnum binaryHeaderEnum)
   at System.Runtime.Serialization.Formatters.Binary.BinaryParser.Run()
   at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(BinaryParser serParser, Boolean fCheck)
   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, Boolean check)
   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream)

My inner most calling code to the framework that's failing is this:

public static object Deserialize(BinaryReader binaryReader)
{
    BinaryReader binaryReader = new BinaryReader(inStream);
    BinaryFormatter binaryFormatter = new BinaryFormatter();
    return binaryFormatter.Deserialize(binaryReader.BaseStream);
}

Thanks in advance.

[danmosemsft]

This is arguably an oversight, as it is really part of Hashtable (it's a private derived type it returns) and is trivial (no fields other than Hashtable).

[danmosemsft]

Several of the non generic collections have Synchronized static methods that return private derived classes. The only one we currently have [Serialized] is SortedList (because it's used in CookieContainer)

That leaves

ArrayList.Synchronized(ArrayList) -- 4%, 17%
ArrayList.Synchronized(IList) -- 0.2%, 3.25%
Hashtable.Synchronized -- 10%
Queue.Synchronized -- 2%
Stack.Synchronized -- 0.8%

The %'s are the usage in the API Compat/selected API Compat corpuses (usage of the static method, we cannot easily tell what sub group of those are ever binary serialized)

[danmosemsft]

Note that (unlike their better known parents) none of these private nested classes had [NonSerialized] on their sync root fields in .NET Framework, so we can't remove those fields in .NET Core if we want them to be serializable (unless we implement ISerializable). Not that it matters.

SyncHashtable actually doesn't have a syncroot field at all, it uses the one on the nested object, which is how the others should have been implemented.

[danmosemsft]

We would not normally mark an object as [Serializable] that has serializable fields of type object, but in these cases we can presumably be confident we know exactly what object is - it's literally object type.

[ershadnozari]

Thanks Dan, for response. The only difference is the hosting console application, Framework vs Core. Can't figure out why this is causing problem. In terms of a potential fix, what can I do as a potential fix to the problem?

[danmosemsft]

@ershadnozari the above were notes to myself 😃

In .NET Core fewer types are binary serializable than in .NET Framework. This is by design because serialization based on BinaryFormatter has historically been fragile and prone to security problems. SyncHashtable is not currently serializable. However we should probably make it so. Meantime you need to avoid serializing it. This may mean using Hashtable and manually locking instead of using SyncHashtable. Or possibly (advanced) using the extension mechanisms of BinaryFormatter to work around the problem.

[jkotas]

Or consider switching to more modern serialization format, like json. Binary serialization is a legacy technology with number of problems. It is frozen in time and it won't evolve.

[ershadnozari]

Thanks to both of you for response. Can’t use JSON serialization as it is xbrl schema taxonomy set, that we are serializing/deserializing for payload validation. I’ll go with what Dan is suggesting. Cheers

[danmosemsft]

@jkotas I'm thinking we should at least mark SyncHashtable with [Serialized] as it has zero fields beyond Hashtable and apparently it's referenced (if not binary serialized) in 10% of APIcompat samples. Perhaps also some of the others (although they do have a sync root field as well, it would be type literal object)

[ershadnozari]

Dan, the libraries are .NET Standard 2.O being called from .NET Framework and Core. There’s no logic in the console apps apart from the execution entry. I’m still confused as why this issue is occurring in the Core app only? Aren’t the set of API’s, in .NET Standard library, the same in both cases?

How will I know/find out wether the change you’re proposing will make it to the framework?

[jkotas]

This is arguably an oversight, as it is really part of Hashtable

It does not look like an oversight to me. All internal parts of Hashtable (Enumerator, KeyCollection, ValueCollection) were serialiable in .NET Framework. None of these parts are serialiable in .NET Core. Same for Dictionary and other collections.

I'm thinking we should at least mark SyncHashtable with [Serialized]

That alone is not sufficient to make SyncHashtable binary serialize correctly. It would also need to implement ISerializable interface because of it inherits from Hashtable that implements ISerializable interface. And there is a good change that there is more to it because of binary serialization is always full of surprises. You won't really know until you add tests.

Aren’t the set of API’s, in .NET Standard library, the same in both cases?

.NET Standard guarantees that the APIs are there, but it does not guarantee that they behave exactly the same.

[ershadnozari]

Appreciate your input gents. What would be an alternative option to BinaryFormatter for serializing object graphs, that works both in Framework and Core? Cheers

[jkotas]

protobuf-net is pretty good.

[danmosemsft]

@ershadnozari I think we can close this now?

[ershadnozari]

Sure, thanks you can close it. I would loved to have a more straight forward to perform to migration. Unfortunately it seems to be a lot more work involved to swap out the serializer

[safern]

Closing per discussion above.