Runtime: Binary Serializing a DataTable with a DataColumn fails with SerializationException

[MaxG117]

Using older and the latest versions of .NET SDK (2.1.4) / .NET Core 2.0.5, the following code:

using System;
using System.Data;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;

namespace MyBinarySerializer
{
    class Program
    {
        static void Main(string[] args)
        {
            DataTable dt = new DataTable("MyTable");
            DataColumn dc = new DataColumn("dc", typeof(Int32));
            dt.Columns.Add(dc);
            dt.RemotingFormat = SerializationFormat.Binary;

            using (FileStream fs = new FileStream("MyTable.bin", FileMode.Create))
            {
                BinaryFormatter bf = new BinaryFormatter();
                bf.Serialize(fs, dt);
            }
        }
    }
}

results in the following exception:

Error Message:
 System.Runtime.Serialization.SerializationException : Type 'System.RuntimeType' in Assembly 'System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e' is not marked as serializable.
Stack Trace:
   at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitSerialize(Object obj, ISurrogateSelector surrogateSelector, StreamingContext context, SerObjectInfoInit serObjectInfoInit, IFormatterConverter converter, ObjectWriterobjectWriter, SerializationBinder binder)
   at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo)
   at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize(Object graph, BinaryFormatterWriter serWriter, Boolean fCheck)
   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph, Boolean check)
   at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph)

I tried a few other types in the DataColumn to no avail. The code works under .NET Framework. It works when the DataTable is empty. It also works if I set dt.RemotingFormat = SerializationFormat.Xml; I've looked at dotnet/runtime#23169 and https://docs.microsoft.com/en-us/dotnet/standard/serialization/binary-serialization, which doesn't list System.Data.DataColumn, but what's the point of serializing DataTable without DataColumn? Any suggestions?

[danmosemsft]

@viktorhofer

[danmosemsft]

DataColumn (and DataRow) are not [Serializable] even in .NET Framework. Instead DataTable organizes their serialization.

This issue is not found by the tests beecause BinaryFormatterTestData.cs is missing dt.RemotingFormat = SerializationFormat.Binary;. By default DataTable binary serializes using XML format - not to be confused with XMLSerializer - and in GetObjectData it sets an string XML blob with info.AddValue.

https://github.com/dotnet/corefx/blob/master/src/System.Data.Common/src/System/Data/DataTable.cs#L232

If you explicitly set SerializationFormat.Binary, it instead does info.AddValue on many individual fields, including one of type Type (Type internally is RuntimeType hence the exception message):

https://github.com/dotnet/corefx/blob/master/src/System.Data.Common/src/System/Data/DataTable.cs#L378

So we did not discover this because of an oversight in the tests.

Our options include

1. Only support Binary Serialization with DataTables set to SerializationFormat.Xml
2. Change how DataTables do SerializationFormat.Binary to serialize Type as a string rather than a Type type, and deserialize with Type.GetType(string) (I'm assuming there aren't more fields after this of problematic type) -- this would break exchnage with other runtimes but it's otherwise appealing
3. Make Type binary serialiable - as I recall we did not do this because we did not have a key scenario but also there was concern this would bring .NET Core some of the problems of desktop serializatoin.

I'm inclined towards dotnet/corefx#2 and require SerializationFormat.Xml for exchange with other runtimes.

@ViktorHofer @stephentoub

[danmosemsft]

@MaxG117 for what reason do you prefer SerializationFormat.Binary on DataTable? Is SerializationFormat.Xml an option?

[MaxG117]

@danmosemsft primarily I'm reporting the bug. I'll let you know the reason on Monday, if I find one.

[MaxG117]

@danmosemsft the reason is: we are testing the compatibility of our Data Provider specific types with all possible use cases.

[ViktorHofer]

Oh I didn't read to the end. Sorry @logiclrd for pinging you. It seems this issue first needs to be discussed before taking any action.

[ViktorHofer]

Make Type binary serialiable - as I recall we did not do this because we did not have a key scenario but also there was concern this would bring .NET Core some of the problems of desktop serializatoin.

That's definitely not an option.

I'm inclined towards dotnet/corefx#2 and require SerializationFormat.Xml for exchange with other runtimes.

I agree. The necessary changes are minor and straightforward to implement. Though without binary serialization hacks we can't support roundtripping with different runtimes (as you already pointed out). This isn't the first case and I'm inclined to just make it work on .NET Core with SerializationFormat.Binary.

[ViktorHofer]

also cc @krwq who helped with DataTable/DataColumn binary serialization.

[danmosemsft]

@ViktorHofer if we're settled on dotnet/corefx#2 then I suggest you give this to @Anipik . ? He can learn about binary serializtaoin.

[ViktorHofer]

@Anipik ping me if you need help