Runtime: Support polymorphic serialization through new option

[steveharter]

Current behavior

Consider a simple model:

abstract class Person {...}
class Customer : Person {...}

The current behavior, by design, is to use the static type and not serialize polymorphically.

// This will serialize all properties on `Person` but not `Customer`:
Person person = new Customer();
string str = JsonSerializer.ToString<Person>(person);

// This does the same thing through generic type inference:
string str = JsonSerializer.ToString(person);

However, if the type is declared to be System.Object then all properties will be serialized:

object person = new Customer();
// This will serialize all properties on `Customer`
string str = JsonSerializer.ToString(person);

In addition, the Type parameter can be specified:

Person person = new Customer();
// This will serialize all properties on `Customer`
string str = JsonSerializer.ToString(person, person.GetType());

All of these semantics are current as will remain as-is.

Proposed API

Add an opt-in setting that will serialize all types polymorphically, instead of just System.Object:

namespace System.Text.Json
{
    public class JsonSerializerOptions // existing class
    {
. . .
        bool SerializePolymorphically { get; set; }    
. . .
    }
}

Example of new behavior:

var options = new JsonSerializerOptions();
options.SerializePolymorphically = true;

// This will serialize all properties on `Customer`:
Person person = new Customer();
string str = JsonSerializer.ToString(person, options);

This new behavior applies both to the root object being serialized (as shown above) plus any properties on POCOs including collections, such as List<Person>.

Note: this issue does not enable any additional support for polymorphic deserialize. Currently, a property or collection of type System.Object will result in a JsonElement being created. Any additional polymorphic deserialize support must be done with a custom converter.

[steveharter]

Moving to future for the following reasons:

• 3.0 schedule is feature-complete in a few days.
• This may not be the best programming model long-term. We should really add feature at the same time we add support for polymorphic deserialization since we may want to share common options or an enum.
• Since we don't support true polymorphic deserialization, polymorphic serialization is not a useful for any round-tripping scenarios.
• The new custom converter feature can be used to handle polymorphic serialization of properties, as well as polymorphic deserialization of properties or types.

[david-driscoll]

I just ran into this scenario when doing some testing with latest preview. Is this something that is documented elsewhere? If not this should be highlighted in the migration guide as a new "feature" for new applications switching to use only the new JSON serializer. Otherwise someone might not notice the subtle change in their applications for a while.

[Symbai]

@steveharter The new custom converter feature cannot be used to handle polymorphic (de)serialization in some cases as, unless I'm missing something, because all properties have to be deserialized by hand / code. Which is not an option for maintainability when you have classes with more than two properties. See https://github.com/dotnet/corefx/issues/39031

The issue is that in the custom converter you only have a Utf8JsonReader which can read but not deserialize. Means you have to read one-by-one and set properties one-by-one.

If there however would be a way to deserialize an object, but only the object not the whole json, within the custom converter all at once, it becomes a feature that can be used.

[steveharter]

@Symbai yes you are correct for deserialization -- there is not an easy way to support without manually authoring.

Here's a test\sample for using a type discriminator to do that:

Also here's a test\sample for polymorphic serialization. Currently the test\sample assumes the base class is abstract.

I'll work on adding real doc and samples shortly.

[Symbai]

@steveharter Thanks but the samples cannot be used in real world scenarios because classes mostly don't have only 2 properties as shown in the sample, but ten to fifty. And now imagine we serialize something that itself has a class as property and this class also has another class and so on. So in the end, we have over hundreds of properties we, currently, have to write for EACH of them this here:

 case "Name of property":
                                ... bla= reader.GetXY();
                                ((MyClass)value).Bla= bla;
                                break;

But if you have to write this more than 10 times already the code gets ugly and hardly maintainable.

Example

public override Person Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
    if (reader.TokenType != JsonTokenType.StartObject)
    {
        throw new JsonException();
    }

    reader.Read();
    if (reader.TokenType != JsonTokenType.PropertyName)
    {
        throw new JsonException();
    }

    string propertyName = reader.GetString();
    if (propertyName != "TypeDiscriminator")
    {
        throw new JsonException();
    }

    reader.Read();
    if (reader.TokenType != JsonTokenType.Number)
    {
        throw new JsonException();
    }

    Person value;
    TypeDiscriminator typeDiscriminator = (TypeDiscriminator)reader.GetInt32();
    switch (typeDiscriminator)
    {
        case TypeDiscriminator.Customer:
            value = new Customer();
            break;

        case TypeDiscriminator.Employee:
            value = new Employee();
            break;

        default:
            throw new JsonException();
    }

    while (reader.Read())
    {
        if (reader.TokenType == JsonTokenType.EndObject)
        {
            return value;
        }

        if (reader.TokenType == JsonTokenType.PropertyName)
        {
            propertyName = reader.GetString();
            reader.Read();
            switch (propertyName)
            {
                case "0":
                    decimal zero = reader.GetDecimal();
                    ((Class1)value).ZERO = zero;
                    break;
                case "1":
                    decimal one = reader.GetDecimal();
                    ((Class1)value).ONE = one;
                    break;
                case "2":
                    decimal two = reader.GetDecimal();
                    ((Class1)value).TWO = two;
                    break;
                case "3":
                    decimal three = reader.GetDecimal();
                    ((Class1)value).THREE = three;
                    break;
                case "4":
                    decimal four = reader.GetDecimal();
                    ((Class1)value).FOUR = four;
                    break;
                case "5":
                    decimal five = reader.GetDecimal();
                    ((Class1)value).FIVE = five;
                    break;
                case "6":
                    decimal six = reader.GetDecimal();
                    ((Class1)value).SIX = six;
                    break;
                case "7":
                    decimal seven = reader.GetDecimal();
                    ((Class1)value).SEVEN = seven;
                    break;
                case "8":
                    decimal eight = reader.GetDecimal();
                    ((Class1)value).EIGHT = eight;
                    break;
                case "9":
                    decimal nine = reader.GetDecimal();
                    ((Class1)value).NINE = nine;
                    break;
                case "10":
                    decimal ten = reader.GetDecimal();
                    ((Class1)value).TEN = ten;
                    break;
                case "11":
                    decimal eleven = reader.GetDecimal();
                    ((Class2)value).ELEVEN = eleven;
                    break;
                case "12":
                    decimal twelve = reader.GetDecimal();
                    ((Class2)value).TWELVE = twelve;
                    break;
                case "13":
                    decimal thirteen = reader.GetDecimal();
                    ((Class2)value).THIRTEEN = thirteen;
                    break;
                case "14":
                    decimal fourteen = reader.GetDecimal();
                    ((Class2)value).FOURTEEN = fourteen;
                    break;
                case "15":
                    decimal fifteen = reader.GetDecimal();
                    ((Class2)value).FIFTEEN = fifteen;
                    break;
                case "16":
                    decimal sixteen = reader.GetDecimal();
                    ((Class2)value).SIXTEEN = sixteen;
                    break;
                case "17":
                    decimal seventeen = reader.GetDecimal();
                    ((Class2)value).SEVENTEEN = seventeen;
                    break;
                case "18":
                    decimal eighteen = reader.GetDecimal();
                    ((Class2)value).EIGHTEEN = eighteen;
                    break;
                case "19":
                    decimal nineteen = reader.GetDecimal();
                    ((Class2)value).NINETEEN = nineteen;
                    break;
                case "20":
                    decimal twenty = reader.GetDecimal();
                    ((Class2)value).TWENTY = twenty;
                    break;
                case "21":
                    decimal twenty-one = reader.GetDecimal();
                    ((Class3)value).TWENTY-ONE = twenty-one;
                    break;
                case "22":
                    decimal twenty-two = reader.GetDecimal();
                    ((Class3)value).TWENTY-TWO = twenty-two;
                    break;
                case "23":
                    decimal twenty-three = reader.GetDecimal();
                    ((Class3)value).TWENTY-THREE = twenty-three;
                    break;
                case "24":
                    decimal twenty-four = reader.GetDecimal();
                    ((Class3)value).TWENTY-FOUR = twenty-four;
                    break;
                case "25":
                    decimal twenty-five = reader.GetDecimal();
                    ((Class3)value).TWENTY-FIVE = twenty-five;
                    break;
                case "26":
                    decimal twenty-six = reader.GetDecimal();
                    ((Class3)value).TWENTY-SIX = twenty-six;
                    break;
                case "27":
                    decimal twenty-seven = reader.GetDecimal();
                    ((Class3)value).TWENTY-SEVEN = twenty-seven;
                    break;
                case "28":
                    decimal twenty-eight = reader.GetDecimal();
                    ((Class3)value).TWENTY-EIGHT = twenty-eight;
                    break;
                case "29":
                    decimal twenty-nine = reader.GetDecimal();
                    ((Class3)value).TWENTY-NINE = twenty-nine;
                    break;
                case "30":
                    decimal thirty = reader.GetDecimal();
                    ((Class3)value).THIRTY = thirty;
                    break;
                case "31":
                    decimal thirty-one = reader.GetDecimal();
                    ((Class4)value).THIRTY-ONE = thirty-one;
                    break;
                case "32":
                    decimal thirty-two = reader.GetDecimal();
                    ((Class4)value).THIRTY-TWO = thirty-two;
                    break;
                case "33":
                    decimal thirty-three = reader.GetDecimal();
                    ((Class4)value).THIRTY-THREE = thirty-three;
                    break;
                case "34":
                    decimal thirty-four = reader.GetDecimal();
                    ((Class4)value).THIRTY-FOUR = thirty-four;
                    break;
                case "35":
                    decimal thirty-five = reader.GetDecimal();
                    ((Class4)value).THIRTY-FIVE = thirty-five;
                    break;
                case "36":
                    decimal thirty-six = reader.GetDecimal();
                    ((Class4)value).THIRTY-SIX = thirty-six;
                    break;
                case "37":
                    decimal thirty-seven = reader.GetDecimal();
                    ((Class4)value).THIRTY-SEVEN = thirty-seven;
                    break;
                case "38":
                    decimal thirty-eight = reader.GetDecimal();
                    ((Class4)value).THIRTY-EIGHT = thirty-eight;
                    break;
                case "39":
                    decimal thirty-nine = reader.GetDecimal();
                    ((Class4)value).THIRTY-NINE = thirty-nine;
                    break;
                case "40":
                    decimal forty = reader.GetDecimal();
                    ((Class4)value).FORTY = forty;
                    break;
                case "41":
                    decimal forty-one = reader.GetDecimal();
                    ((Class5)value).FORTY-ONE = forty-one;
                    break;
                case "42":
                    decimal forty-two = reader.GetDecimal();
                    ((Class5)value).FORTY-TWO = forty-two;
                    break;
                case "43":
                    decimal forty-three = reader.GetDecimal();
                    ((Class5)value).FORTY-THREE = forty-three;
                    break;
                case "44":
                    decimal forty-four = reader.GetDecimal();
                    ((Class5)value).FORTY-FOUR = forty-four;
                    break;
                case "45":
                    decimal forty-five = reader.GetDecimal();
                    ((Class5)value).FORTY-FIVE = forty-five;
                    break;
                case "46":
                    decimal forty-six = reader.GetDecimal();
                    ((Class5)value).FORTY-SIX = forty-six;
                    break;
                case "47":
                    decimal forty-seven = reader.GetDecimal();
                    ((Class5)value).FORTY-SEVEN = forty-seven;
                    break;
                case "48":
                    decimal forty-eight = reader.GetDecimal();
                    ((Class5)value).FORTY-EIGHT = forty-eight;
                    break;
                case "49":
                    decimal forty-nine = reader.GetDecimal();
                    ((Class5)value).FORTY-NINE = forty-nine;
                    break;
                case "50":
                    decimal fifty = reader.GetDecimal();
                    ((Class5)value).FIFTY = fifty;
                    break;
                case "51":
                    decimal fifty-one = reader.GetDecimal();
                    ((Class6)value).FIFTY-ONE = fifty-one;
                    break;
                case "52":
                    decimal fifty-two = reader.GetDecimal();
                    ((Class6)value).FIFTY-TWO = fifty-two;
                    break;
                case "53":
                    decimal fifty-three = reader.GetDecimal();
                    ((Class6)value).FIFTY-THREE = fifty-three;
                    break;
                case "54":
                    decimal fifty-four = reader.GetDecimal();
                    ((Class6)value).FIFTY-FOUR = fifty-four;
                    break;
                case "55":
                    decimal fifty-five = reader.GetDecimal();
                    ((Class6)value).FIFTY-FIVE = fifty-five;
                    break;
                case "56":
                    decimal fifty-six = reader.GetDecimal();
                    ((Class6)value).FIFTY-SIX = fifty-six;
                    break;
                case "57":
                    decimal fifty-seven = reader.GetDecimal();
                    ((Class6)value).FIFTY-SEVEN = fifty-seven;
                    break;
                case "58":
                    decimal fifty-eight = reader.GetDecimal();
                    ((Class6)value).FIFTY-EIGHT = fifty-eight;
                    break;
                case "59":
                    decimal fifty-nine = reader.GetDecimal();
                    ((Class6)value).FIFTY-NINE = fifty-nine;
                    break;
                case "60":
                    decimal sixty = reader.GetDecimal();
                    ((Class6)value).SIXTY = sixty;
                    break;
                case "61":
                    decimal sixty-one = reader.GetDecimal();
                    ((Class7)value).SIXTY-ONE = sixty-one;
                    break;
                case "62":
                    decimal sixty-two = reader.GetDecimal();
                    ((Class7)value).SIXTY-TWO = sixty-two;
                    break;
                case "63":
                    decimal sixty-three = reader.GetDecimal();
                    ((Class7)value).SIXTY-THREE = sixty-three;
                    break;
                case "64":
                    decimal sixty-four = reader.GetDecimal();
                    ((Class7)value).SIXTY-FOUR = sixty-four;
                    break;
                case "65":
                    decimal sixty-five = reader.GetDecimal();
                    ((Class7)value).SIXTY-FIVE = sixty-five;
                    break;
                case "66":
                    decimal sixty-six = reader.GetDecimal();
                    ((Class7)value).SIXTY-SIX = sixty-six;
                    break;
                case "67":
                    decimal sixty-seven = reader.GetDecimal();
                    ((Class7)value).SIXTY-SEVEN = sixty-seven;
                    break;
                case "68":
                    decimal sixty-eight = reader.GetDecimal();
                    ((Class7)value).SIXTY-EIGHT = sixty-eight;
                    break;
                case "69":
                    decimal sixty-nine = reader.GetDecimal();
                    ((Class7)value).SIXTY-NINE = sixty-nine;
                    break;
                case "70":
                    decimal seventy = reader.GetDecimal();
                    ((Class7)value).SEVENTY = seventy;
                    break;
                case "71":
                    decimal seventy-one = reader.GetDecimal();
                    ((Class8)value).SEVENTY-ONE = seventy-one;
                    break;
                case "72":
                    decimal seventy-two = reader.GetDecimal();
                    ((Class8)value).SEVENTY-TWO = seventy-two;
                    break;
                case "73":
                    decimal seventy-three = reader.GetDecimal();
                    ((Class8)value).SEVENTY-THREE = seventy-three;
                    break;
                case "74":
                    decimal seventy-four = reader.GetDecimal();
                    ((Class8)value).SEVENTY-FOUR = seventy-four;
                    break;
                case "75":
                    decimal seventy-five = reader.GetDecimal();
                    ((Class8)value).SEVENTY-FIVE = seventy-five;
                    break;
                case "76":
                    decimal seventy-six = reader.GetDecimal();
                    ((Class8)value).SEVENTY-SIX = seventy-six;
                    break;
                case "77":
                    decimal seventy-seven = reader.GetDecimal();
                    ((Class8)value).SEVENTY-SEVEN = seventy-seven;
                    break;
                case "78":
                    decimal seventy-eight = reader.GetDecimal();
                    ((Class8)value).SEVENTY-EIGHT = seventy-eight;
                    break;
                case "79":
                    decimal seventy-nine = reader.GetDecimal();
                    ((Class8)value).SEVENTY-NINE = seventy-nine;
                    break;
                case "80":
                    decimal eighty = reader.GetDecimal();
                    ((Class8)value).EIGHTY = eighty;
                    break;
                case "81":
                    decimal eighty-one = reader.GetDecimal();
                    ((Class9)value).EIGHTY-ONE = eighty-one;
                    break;
                case "82":
                    decimal eighty-two = reader.GetDecimal();
                    ((Class9)value).EIGHTY-TWO = eighty-two;
                    break;
                case "83":
                    decimal eighty-three = reader.GetDecimal();
                    ((Class9)value).EIGHTY-THREE = eighty-three;
                    break;
                case "84":
                    decimal eighty-four = reader.GetDecimal();
                    ((Class9)value).EIGHTY-FOUR = eighty-four;
                    break;
                case "85":
                    decimal eighty-five = reader.GetDecimal();
                    ((Class9)value).EIGHTY-FIVE = eighty-five;
                    break;
                case "86":
                    decimal eighty-six = reader.GetDecimal();
                    ((Class9)value).EIGHTY-SIX = eighty-six;
                    break;
                case "87":
                    decimal eighty-seven = reader.GetDecimal();
                    ((Class9)value).EIGHTY-SEVEN = eighty-seven;
                    break;
                case "88":
                    decimal eighty-eight = reader.GetDecimal();
                    ((Class9)value).EIGHTY-EIGHT = eighty-eight;
                    break;
                case "89":
                    decimal eighty-nine = reader.GetDecimal();
                    ((Class9)value).EIGHTY-NINE = eighty-nine;
                    break;
                case "90":
                    decimal ninety = reader.GetDecimal();
                    ((Class9)value).NINETY = ninety;
                    break;
                case "91":
                    decimal ninety-one = reader.GetDecimal();
                    ((Class10)value).NINETY-ONE = ninety-one;
                    break;
                case "92":
                    decimal ninety-two = reader.GetDecimal();
                    ((Class10)value).NINETY-TWO = ninety-two;
                    break;
                case "93":
                    decimal ninety-three = reader.GetDecimal();
                    ((Class10)value).NINETY-THREE = ninety-three;
                    break;
                case "94":
                    decimal ninety-four = reader.GetDecimal();
                    ((Class10)value).NINETY-FOUR = ninety-four;
                    break;
                case "95":
                    decimal ninety-five = reader.GetDecimal();
                    ((Class10)value).NINETY-FIVE = ninety-five;
                    break;
                case "96":
                    decimal ninety-six = reader.GetDecimal();
                    ((Class10)value).NINETY-SIX = ninety-six;
                    break;
                case "97":
                    decimal ninety-seven = reader.GetDecimal();
                    ((Class10)value).NINETY-SEVEN = ninety-seven;
                    break;
                case "98":
                    decimal ninety-eight = reader.GetDecimal();
                    ((Class10)value).NINETY-EIGHT = ninety-eight;
                    break;
                case "99":
                    decimal ninety-nine = reader.GetDecimal();
                    ((Class10)value).NINETY-NINE = ninety-nine;
                    break;

            }
        }
    }

    throw new JsonException();
}

[AlexanderJohnston]

This seriously needs to be documented somewhere, like on the System.Text.Json namespace, in highlighted text. I spent hours trying to track down why it was serializing empty strings while working in a Blazor project.

[knom]

This is bad.. just ran into this as well.. after hours of debugging :-(
Is there a workaround using Newtonsoft.JSON?

[steentottrup]

My own findings:

https://stackoverflow.com/questions/58305928/the-new-asp-net-core-3-0-json-serializer-is-leaving-out-data/58316304#58316304

And an article on how to "go back" to Newtonsoft.Json:

https://devblogs.microsoft.com/dotnet/try-the-new-system-text-json-apis/

[Symbai]

Is there a workaround using Newtonsoft.JSON?

On Newtonsoft it's super easy. For example just tell Newtonsoft all known Types and Newtonsoft does all the magic itself for you. No converter, no worries:

public static readonly JsonSerializerSettings Settings = new JsonSerializerSettings {
    TypeNameHandling = TypeNameHandling.Objects,
    SerializationBinder = new KnownTypesBinder {
        KnownTypes = new List<Type> {
            typeof(TypeA),
            typeof(TypeB),
            typeof(TypeC),
            typeof(TypeD),
            typeof(TypeE)
        }
}};

I really wish this would get more attention, at least set to 5.0 milestone instead of future. Currently it's just unusable for polymorphic situations and there is NO workaround (I consider the converter as no workaround in real world scenarios for the reason I posted earlier), so switching is not possible.

[steveharter]

Moving to 5.0.

However this should also be tied to polymorphic deserialization support as well which is also a requested feature but much larger in scope. There are several requests for that and the implementation must be secure and opt-in through known types; we need to combine\triage these polymorphic issues -- here's one request: https://github.com/dotnet/corefx/issues/41347

[bklooste]

Known types / magic polymorphism on the wire are bad patterns and I have seen people waste a ton of time on weird issues and bugs around this - the benefit aint worth it. Neither Json or XML are polymorphic by design and making them do it is a hack at best.

That said we should document explicit type creation better and possibly have some patterns around this eg

var typeName = json.GetProperty("TypeName").GetString(); var type = Type.GetType(typeName); return (T) JsonSerializer.Deserialize(utfJson , type);

[Symbai]

@bklooste newtonsoft Json supports it, without weird issues and bugs around. Since the .NET Team is working together with the author of Newtonsoft Json, it's even more easier.

And regarding XML, most of its code was written for more than a decade ago and remains untouched. Obviously it lacks of features and therefore is a bad comparison.

[liesahead]

As I needed only one-side serializer for specific base class (to make API return derived classes properties), I came up with current solution

public class CustomConverter : JsonConverter<BaseClass>
{
    private readonly JsonSerializerOptions _serializerOptions;

    public CustomConverter()
    {
        _serializerOptions = new JsonSerializerOptions
        {
            PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
            IgnoreNullValues = true,
        };
    }

    public override bool CanConvert(Type objectType)
    {
        return (objectType == typeof(BaseClass));
    }

    public override BaseClass Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        throw new NotImplementedException();
    }

    public override void Write(Utf8JsonWriter writer, BaseClass value, JsonSerializerOptions options)
    {
        writer.WriteStringValue(JsonSerializer.SerializeToUtf8Bytes(value, value.GetType(), _serializerOptions));
    }
}

[sfadeev]

@DenisSemionov, it seems like your sample escapes classes and writes data as string. To serialize normally json converter should be something like this (similar to @steveharter sample above):

public class PolymorphicWriteOnlyJsonConverter<T> : JsonConverter<T>
{
    public override T Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        throw new NotImplementedException();
    }

    public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options)
    {
        JsonSerializer.Serialize(writer, value, value.GetType(), options);
    }
}

... and then registered in JsonOptions:

options.JsonSerializerOptions.Converters.Add(new PolymorphicWriteOnlyJsonConverter<BaseClass>());

[liesahead]

@sfadeev , I agree with you, noticed my mistake too late. Can't find any good examples or documentation for polymorphic serialization in .net core 3.0, need to support 9 subclasses in one web API get method. I will share my solution if I come up with a smart one.

[sfadeev]

need to support 9 subclasses in one web API get method. I will share my solution if I come up with a smart one.

@DenisSemionov, you can use sample from my comment above.

[liesahead]

need to support 9 subclasses in one web API get method. I will share my solution if I come up with a smart one.

@DenisSemionov, you can use sample from my comment above.

@sfadeev, This works exactly as needed, much appreciate for provided example!

[philpowers]

I also ran into this issue, which seemed very counter-intuitive to me. I understand the issues with deserializing polymorphic types, but serialization should be trivial because the types are _always known_ at runtime. Because of that, I somewhat disagree that this should be tied to deserialization support (which I expected to write custom code for anyway).

Anyway, I have dealt with this by making a more generic version of the solution given by @sfadeev :

    public class AbstractWriteOnlyJsonConverter : JsonConverter<object>
    {
        public override bool CanConvert(Type typeToConvert)
        {
            return typeToConvert.IsAbstract;
        }

        public override object Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            throw new NotImplementedException();
        }

        public override void Write(Utf8JsonWriter writer, object value, JsonSerializerOptions options)
        {
            JsonSerializer.Serialize(writer, value, value.GetType(), options);
        }
    }

This allows the serializer to serialize any object derived from an abstract class:

public abstract class DtoDetails
{
}

public class ConcreteDetails : DtoDetails
{
    public string ConcreteField1 { get; set; }
    // ...
}

public class RequestDto
{
    public DtoDetails Details { get; set; }   
}

Details property of RequestDto can now be serialized properly:

options.Converters.Add(new AbstractWriteOnlyJsonConverter());

// ...
var requestDto = new RequestDto { /* ... */ }
var serialized = JsonSerializer.Serialize(requestDto, options);

This has so far worked for me.

[GarrettClyde]

We're going back to Newtonsoft for now. Here's a pretty unsophisticated workaround that might help someone else out.

    public static class RealJsonSerializer
    {
        public static string Serialize<TValue>(TValue value, JsonSerializerOptions options = null)
        {
            object objectValue = value;
            return JsonSerializer.Serialize(objectValue, options);
        }  
    }

[y-lobau]

I've also lost half a day on that debugging asp.net sources. As Newtonsoft's out-of-the-box behavior covers inheritence, i consider this as a real breaking change which will definitely lead to regression after migration to 3.0. I guess the 1st best is to make a red alert in corresponding section of the article Migrate from ASP.NET Core 2.2 to 3.0.

[lwansbrough]

+1 for a day lost to this. It's not just polymorphic class abstractions, it happens to interfaces too.

If you make an interface IFoo with property IBar, which is implemented by Foo : IFoo, and then populate IBar with Baz : IBar, when you serialize Foo, all Baz properties are dropped except for those of the IBar interface. That's totally unpractical and I can't imagine why you'd find it appropriate to recommend switching from Newtonsoft with such a massive breaking change left undocumented.

I would go as far as saying making polymorphic serialization _optional_, and then _not making it the default_ and then also _not including it in 3.0_ should be classified as a bug.

[jan-johansson-mr]

I followed your example, and I got the exception: System.NotSupportedException: 'Deserialization of interface types is not supported' from System.Text.Json. Is this how you set up the types?

    public interface IFoo
    {
        string Label { get; set; }
        IBar Bar { get; set; }
    }

    public interface IBar
    {
        int Age { get; set; }
        string Name { get; set; }
    }
    public class Foo : IFoo
    {
        public string Label { get; set; }
        public IBar Bar { get; set; }
    }
    public class Bar : IBar
    {
        public int Age { get; set; }
        public string Name { get; set; }
    }

As an example...

One observation I've made is that only properties are handled by System.Text.Json, e.g. if you have a type with fields, then the fields will not be handled at all.

[ahsonkhan]

From @mauricio-bv in https://github.com/dotnet/runtime/issues/31742

I am trying to use the System.Text.Json serialization libray, but I can't get it is serialize it as I used to with Newtonson. Properties in derived classes or Interfaces are not serializing. This issue has also been asked and explained in the link below.

Link

Is there any serialization option I should set for proper serialization?

cc @Arash-Sabet, @bailei1987, @bstordrup

[rosieks]

It would be nice if this could be compatible with OpenAPI polymorphism: https://swagger.io/docs/specification/data-models/inheritance-and-polymorphism/

[ttrinidad]

The solution for deep serialization of derived types, (and many other amenities) is to keep using Newtonsoft, which work as desired by most reasonable people

Or do as the docs recommend and change your classes property types to "object":
https://docs.microsoft.com/en-us/dotnet/standard/serialization/system-text-json-how-to#serialize-properties-of-derived-classes

We get it, it's free, but it's just not good as advertised.

[jan-johansson-mr]

Hi,

It's not that hard to add polymorphic support to System.Text.Json via some Converter. However, once done we know we break JSON specification, since JSON specification doesn't support metadata anyway (we can however add some ad hoc convention for metadata). So the good news is, yes, it's very possible to add polymorphic support, and it is not that hard either.

The bad news is that by design ASP.NET Core endpoint will not support polymorphism anyway. I'm including some examples for illustration. Assume two types, the base type and the type inheriting from the base type, called Payload.

If we feed the instance of the polymorphed object to the serializer as

JsonSerializer.Serialize(payload, _jsonSerializerOptions)

Then we get the following output with the instantiating type embedded in the JSON document:

{"$i":"JsonSerializationTest.Payload","title":"Title","description":"Description","id":"72f191f9-35b8-459e-897b-98745d4f4005"}

If we feed the same object with slightly tweaked code

JsonSerializer.Serialize(payload, payload.GetType(), _jsonSerializerOptions)

Then we get the output

{"title":"Title","description":"Description","id":"72f191f9-35b8-459e-897b-98745d4f4005"}

The effect is that yes, we kind of get the same "polymorphed" object on top level, but we've lost the polymorph metadata in the process (the $i tag in the JSON document).

Notice the payload.GetType() as parameter to System.Text.Json.JsonSerializer? That is the reason for the difference in the output.

Look at the code in ASP.NET project, from line 77 (where we get the type) and then further down where the serialization actually occurs. If you read the comment, you also see that this is by design, and of course only applies to the top level of the object.

Now, if I add my [polymorphic] Converter to System.Text.Json in the ASP.NET Core, e.g.

    .AddControllers()
    .AddJsonOptions(configure => ...)

I'll get exactly the same behavior as described above. But if I feed my object in some deeper level, e.g. I return new { payload = payload }, then my Converter kicks in and I get the expected output (however, now it's on second level and not first and that's inconvenient).

I understand the reason for this design in the ASP.NET Core, but again, even if they would build in support for polymorphic serialization with System.Text.Json, the same problem that I described will hit you anyway because of this design.

[lwansbrough]

@jan-johansson-mr IMO any performance gains made from stripping expected behaviour from .NET’s defacto JSON library is a disingenuous comparison. (Performance was the goal of this library if I recall correctly.) It’s very easy to make anything fast if we rip out all the functionality. Fact is, System.Text.Json breaks behaviour and expectations around serialization in the .NET world. .NET is popular because for the most part it just works. From that perspective, this library has dysfunctional behaviour. And it definitely should not be advertised as a cost-free drop in replacement for JSON.NET.

[sake402]

@jan-johansson-mr I second @lwansbrough with his comment.
Worse now is if given

public class Base{
public string PropertyBase{ get; set; }="Base";
}
public class Foo:Base{
public string PropertyFoo{ get; set; } = "Foo";
}

public class Bar:Base{
public string PropertyBar{ get; set; } = "Bar";
}

If we then do

var bases = new[]{ new Foo(), new Bar() }
JsonSerializer.Serialize<object>(bases);

We still get

[{"PropertyBase":"Base"}, {"PropertyBase":"Base"}]

which entirely strip off upper class hierachy property yet. Despite using object as the serialization type

[TheBuzzSaw]

@sake402 While not a solution, does it start working if you cast the array itself to object[]?

[qsdfplkj]

There is another issue that adds metadata to prevent infinite loops. JsonSerializerOptions.ReferenceHandler. If we allow metadata we migh also allow type information to be included and use this during deserialization.

Besides the SerializePolymorphically option you could also tweak the model with attributes to only make certain properties polymorphic [SerializePolymorphically] or use a convention that properties declared as interfaces (or abstract classes) are always polymorphic even when the SerializePolymorphically option is disabled. You could also skip the metadata $type when the declared type equals the runtime type when SerializePolymorphically option is enabled so that there is no difference in the result json output

Personally I think if you serialize $type metadata only when the declared type differs from the runtime type you don't even need a SerializePolymorphically option where this wouldn't break existing scenario's because I _assume_ that the declared type should equal the runtime type.

Security:
If deserialized with a generic type argument it will fail due to casting but if deserialized with the generic type argument being object you should throw an not supported exception because you might have a security vulnerability.

[qsdfplkj]

I use the following polymorphic json converter factory which serializes interfaces and abstract classes with type metadata and uses that for deserialization. (See also my previous comment for my thoughts on polymorphic serialization.)

This converter serializes an object or property that is declared as abstract or interface as follows:
{ $type: typeof(T).AssemblyQualifiedName, $value: { propertyName: propertyValue } }

It would be nice though if it worked like this:
{ $type: typeof(T).AssemblyQualifiedName, propertyName: propertyValue }

The difference is that the first $type is property about $value the second $type is a property of $value it itself. The later I think is cleaner and is the way Newtonsoft seems to do it. For it to work like that I would need to loop over the properties when (de)serializing myself.

public class PolymorphicJsonConverter<T> : JsonConverter<T>
{
    public override bool CanConvert(Type typeToConvert)
    {
        return typeToConvert.IsAbstract; && typeof(T) == typeToConvert;
    }

    public override object Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        reader.Read();
        reader.Read();
        string assemblyQualifiedName = reader.GetString();
        reader.Read();
        T result = (T)JsonSerializer.Deserialize(ref reader, Type.GetType(assemblyQualifiedName), options);
        reader.Read();
        return result;
    }

    public override void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options)
    {
        writer.WriteStartObject();
        writer.WritePropertyName("$type");
        writer.WriteStringValue(value.GetType().AssemblyQualifiedName);
        writer.WritePropertyName("$value");
        JsonSerializer.Serialize(writer, value, value.GetType(), options);
        writer.WriteEndObject();
    }
}

public class PolymorphicJsonConverter : JsonConverterFactory
{
    public override bool CanConvert(Type typeToConvert)
    {
        return typeToConvert.IsAbstract && !typeToConvert.IsGenericType;
    }

    public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options)
    {
        Type genericType = typeof(PolymorphicJsonConverter<>).MakeGenericType(typeToConvert);
        return (JsonConverter)Activator.CreateInstance(genericType, false);
    }
}

[qsdfplkj]

If you like OOP you should be serializing IList and not a List. I don’t have a problem to serialize object[] but we need metadata to deserialize the Json but there is no metadata yet so deserialize should be called with the same generic type argument as serialize (which only works for the root object and not for properties) When you use object or the non generic method you cannot deserialize without metadata. Neither can json.net by default but they have metadata support so enable it and it works. Besides this I noticed that both serializers don’t support explicit interfaces. I don’t think this metadata will degrade performance but I don’t like metadata when it isn’t needed (which json.net generates). I personally don’t understand why we use JavaScript a dynamic typed language to serialize strong type languages and then complain about stuff not working or not according to “the standard”. Was it never ment to be send anywhere but to JavaScript?

[lukacs-andras]

@jan-johansson-mr I second @lwansbrough with his comment.
Worse now is if given

public class Base{
public string PropertyBase{ get; set; }="Base";
}
public class Foo:Base{
public string PropertyFoo{ get; set; } = "Foo";
}

public class Bar:Base{
public string PropertyBar{ get; set; } = "Bar";
}

If we then do

var bases = new[]{ new Foo(), new Bar() }
JsonSerializer.Serialize<object>(bases);

We still get

[{"PropertyBase":"Base"}, {"PropertyBase":"Base"}]

which entirely strip off upper class hierachy property yet. Despite using object as the serialization type

In that case, the correct generic type parameter is object[] instead of object.

[steveharter]

FFS, just own up to a bad decision and make it the default behavior, like every other worthwhile serialization tool out there. The thing needs to do before it fasts. As far as stupid default behaviors go, this has got to be one of the dumbest I've come across. A serialization library that (by default) doesn't serialize what you give it. Awesome! No wonder it's so fast!

The first release in 3.0 was considered a minimum viable product and getting polymorphism in correctly wasn't feasible. Polymorphic serialization is easy -- we already do it when type == typeof(object) -- but adding the metadata for deserialization needs to be done right where it uses "known types" and not the raw CLR string (for security). This feature didn't fit into the 5.0 release either.

Polymorphic serialization of a root type has basically zero perf overhead -- calling object.GetType(). Polymorphic deserialization will have a perf hit if the metadata is there, but that entails a simple lookup from the string-based metadata ("$type:MyKnownType") to a CLR Type.

System.Text.Json was never intended to replace Newtonsoft.JSON for all scenarios. From the documention:

_System.Text.Json focuses primarily on performance, security, and standards compliance. It has some key differences in default behavior and doesn't aim to have feature parity with Newtonsoft.Json. For some scenarios, System.Text.Json has no built-in functionality, but there are recommended workarounds. For other scenarios, workarounds are impractical. If your application depends on a missing feature, consider filing an issue to find out if support for your scenario can be added._

[Symbai]

System.Text.Json was never intended to replace Newtonsoft.JSON for all scenarios

If you dont support polymorphic (de)serialization then the number of scenarios are extremely small makes me wonder why doing all the afford of creating it then? Its like you build a calculator and tell us "Hey this is a super fast calculator you should use for your mathematical operations" and on usage it turns out the calculator supports subtract, add, divide but not multiply. And on request you tell us "Well, it does not support multiply thats true. But our calculator was not supposed to calculate in every scenario 🤷‍♂️"

Using interfaces, abstract classes, and inherits is a huge part of .NET and its currently not supported, not even in .NET5. Its like a calculator which cannot multiply. And this is exactly how it feels like. I really hope this feature gets higher priority and perhaps a backport to .NET5.

[brunoalreis]

Is the future .net api's expected to not work by default and, the only solutions, are workarounds?

I start using .net core 1.1, then migrate to 2.1 and now I am migrating to 3.1. Of all migrations, this one is being a nightmare.
First I start with https://github.com/dotnet/runtime/issues/29932 problem. Ok, exists a workaround. Now this?

If System.Text.Json is still in development, I understand. But please don't make it the default serializer for api's. This is like creating a super engine that only work if it is a sunny day. In an OOP this must be possible.

Just sharing one of my real world scenarios:
My dashboard, has a list of widgets. I have many types of widgets, with different properties, but all widgets extends the Widget abstract class.

For the sake of simplicity:

public abstract class Widget{
  public decimal Height {...}
  public decimal Width {...}
}

public class Dashboard {
  List<Widget> Widgets {...}
}

public class Widget1: Widget {
  public string Widget1Data {...}
}

public class Widget2: Widget {
  public string[] Widget2Data {...}
  public OtherClass Widget2DataOther {...}
}

How can my api return the complete information of one dashboard, including widgets? Can't. Everything is broken. Have to use other workaround or .AddNewtonsoftJson(). Choosing last one for now.

Hope you change your mind. An if your going to make something important like this to be used by default, make it "for all scenarios". If not the default behavior, at least, make it optional. If Its not ready for real case scenarios, keep working one as default. This is a must have, not a should have.

[ColinM9991]

Just chiming in to add my voice to the weight on one side.

I am sorry for saying so but this was a very poor design choice and it would be good if Microsoft could refrain from assuming what is wanted or needed of APIs and technologies (purely from a consumer standpoint) that people use on a daily basis. JSON is at the very heart of hundreds of thousands, if not millions, of public APIs.

There is a bit of a contradictory statement that System.Text.Json is not a replacement of Newtonsoft.Json yet all Microsoft documentation that I come across, around System.Text.Json, seems to enjoy pointing out the flaws and issues with Newtonsoft, which pushes people down a path of using System.Text.Json.

I read elsewhere (I believe the Microsoft docs) that polymorphic serialization is not supported because it 'leaks' information that shouldn't be leaked. Isn't that the purpose of JsonIgnoreAttribute? I shouldn't have to create a JsonConverter<T> or writer to clearly portray my intentions when OOP exists for that purpose.

[oliverjanik]

I'm writing a new service in .NET 5 and this issue is still present.

I have a tree of objects with nested properties declared as interfaces. The serialiser just skips all of them, issues empty object.

What a nightmare, this is one of those instances where you can tell the disconnect between designers at Microsoft and real use-case scenarios.

I understand completely that polymorphic deserialisation is a tough problem. But polymorphic serialisation is a straight-forward decision.