Runtime: New Asp.NET Core 3.0 Json doesn't serialize Dictionary<Key,Value>

[willyt150]

.NET Core 3.0 Preview 7

Asp.NET Web Apis, when returning a Dictionary it fails with a NotSupportedException. I've included the exception below.

Also, the ControllerBase.BadRequest method takes in a ModelStateDictionary, but when that's returned the serializer blows up as well with a NotSupportedException, but a slightly different message.

When will this support be added? Since this has been supported in Json.net and other serializers for a while I hope this is on the radar.

I do appreciate the fact that I can opt back in to using Json.net, so thank you very much for that!

Exception when returning a Dictionary
System.NotSupportedException: The collection type 'System.Collections.Generic.Dictionary`2[System.Int32,System.String]' is not supported.
at System.Text.Json.JsonClassInfo.GetElementType(Type propertyType, Type parentType, MemberInfo memberInfo, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.CreateProperty(Type declaredPropertyType, Type runtimePropertyType, PropertyInfo propertyInfo, Type parentClassType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddProperty(Type propertyType, PropertyInfo propertyInfo, Type classType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddPolicyProperty(Type propertyType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo..ctor(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializerOptions.GetOrAddClass(Type classType)
at System.Text.Json.WriteStackFrame.Initialize(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializer.WriteAsyncCore(Stream utf8Json, Object value, Type type, JsonSerializerOptions options, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|21_0(ResourceInvoker invoker, IActionResult result)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|29_0TFilter,TFilterAsync
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNextTFilter,TFilterAsync
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|17_1(ResourceInvoker invoker)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

Exception when returning BadRequest
System.NotSupportedException: The collection type 'Microsoft.AspNetCore.Mvc.SerializableError' is not supported.
at System.Text.Json.JsonClassInfo.GetElementType(Type propertyType, Type parentType, MemberInfo memberInfo, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.CreateProperty(Type declaredPropertyType, Type runtimePropertyType, PropertyInfo propertyInfo, Type parentClassType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddProperty(Type propertyType, PropertyInfo propertyInfo, Type classType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddPolicyProperty(Type propertyType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo..ctor(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializerOptions.GetOrAddClass(Type classType)
at System.Text.Json.WriteStackFrame.Initialize(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializer.WriteAsyncCore(Stream utf8Json, Object value, Type type, JsonSerializerOptions options, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|21_0(ResourceInvoker invoker, IActionResult result)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|29_0TFilter,TFilterAsync
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNextTFilter,TFilterAsync
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|17_1(ResourceInvoker invoker)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)

[ahsonkhan]

Both "not supported exception" errors are limitations within the built-in serializer and is by design (at least for what is shipping in 3.0).

When will this support be added? Since this has been supported in Json.net and other serializers for a while I hope this is on the radar.
I do appreciate the fact that I can opt back in to using Json.net, so thank you very much for that!

There a bunch of serializer capabilities that are on our radar to support in vNext (so 5.0) and beyond, with custom dictionary support being one of them.

Asp.NET Web Apis, when returning a Dictionary it fails with a NotSupportedException

When serializing, only Dictionary<string, TValue> is supported today (i.e. TKeys that are string). Your dictionary is of <int, string> which isn't supported.
https://github.com/dotnet/corefx/blob/93d7aa1c1737b6da29d04b78557215e18eb786d4/src/System.Text.Json/tests/Serialization/DictionaryTests.cs#L385-L390

@steveharter, @layomia - is there a potential workaround here in the meantime? What would it take to add support for non-string key'd dictionary within the serializer for 5.0?

System.NotSupportedException: The collection type 'Microsoft.AspNetCore.Mvc.SerializableError' is not supported.

~Adding support for a type like SerializableError was not on my radar. @pranavkm, @rynowak - what's the context here? I am not familiar with ModelStateDictionary, could this be supported within mvc itself with a custom converter?~

Edit: Nevermind, that's already fixed.

[ahsonkhan]

System.NotSupportedException: The collection type 'Microsoft.AspNetCore.Mvc.SerializableError' is not supported.

This was a known issue https://github.com/aspnet/AspNetCore/issues/11459 that was recently fixed (as part of preview 8): https://github.com/dotnet/corefx/pull/39001

[willyt150]

Thank you very much for your quick replies @ahsonkhan!

The "limitation" of the key being a string actually makes sense when I think about it. I see now that Json.net actually generates json with the key being a string, when deserializing it would just get me an int back. It would definitely be nice having the support for non-string keys in the future, but not a show stopper.

Ok, glad to hear that the Mvc.SerializableError not being supported has been fixed. Any ideas on if there's a hoped for release date of Preview 8? Tried to search and find something, but not seeing anything about that.

Once preview 8 comes out we'll try giving the .net core 3 json serialization library a try again, but for now we're needing to stick with Json.net

[layomia]

@steveharter, @layomia - is there a potential workaround here in the meantime? What would it take to add support for non-string key'd dictionary within the serializer for 5.0?

>

@ahsonkhan @willyt150 the workaround for this is to use a custom converter that implements JsonConverter<T> where T is Dictionary<int, string>.
See https://github.com/dotnet/corefx/issues/36639#issue-429928740 for some examples.

[ahsonkhan]

Any ideas on if there's a hoped for release date of Preview 8?

Some time later this month.

Thinking about this some more, removing the up-for-grabs for now since it may be something we don't want to support by default.

[willyt150]

Thank you @layomia I'll take a look into that.

Thank you @ahsonkhan, looking forward to the fix coming through!

[ahsonkhan]

From @namse (from https://github.com/dotnet/corefx/issues/40404):

Hello. When I try to serialize Dictionary with integer key, it throw System.NotSupportedException.

I think it makes sense to support Json serialization which Dictionary has ToString-able key. for example, when we run ToString for int or boolean, it return "123" or "true". I think that key is ToString-able key.

Verison

System.Text.Json Nuget Version : 4.6.0-preview8.19405.3

Code

var dictionary = new Dictionary<int, string>
{
  [5] = "five"
};
JsonSerializer.Serialize(dictionary);

Expected

"{"5": "five"}"

But what happen

Error System.NotSupportedException Thrown

Actually, there is compatibility problem when I change Newtonsoft.Json to System.Text.Json. They return string as I expected. I think System.Text.Json don't have to be compatible but... you know.

[hez2010]

I have implemented a converter which support both Serialization and Deserialization for IDictionary<TKey, TValue> where TKey has a static method TKey Parse(string):

using System;
using System.Collections;
using System.Collections.Generic;
using System.Globalization;
using System.Reflection;
using System.Text.Json;
using System.Text.Json.Serialization;

namespace JsonDictionaryConverter
{
    sealed class JsonNonStringKeyDictionaryConverter<TKey, TValue> : JsonConverter<IDictionary<TKey, TValue>>
    {
        public override IDictionary<TKey, TValue> Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            var convertedType = typeof(Dictionary<,>)
                .MakeGenericType(typeof(string), typeToConvert.GenericTypeArguments[1]);
            var value = JsonSerializer.Deserialize(ref reader, convertedType, options);
            var instance = (Dictionary<TKey, TValue>)Activator.CreateInstance(
                typeToConvert,
                BindingFlags.Instance | BindingFlags.Public,
                null,
                null,
                CultureInfo.CurrentCulture);
            var enumerator = (IEnumerator)convertedType.GetMethod("GetEnumerator")!.Invoke(value, null);
            var parse = typeof(TKey).GetMethod("Parse", 0, BindingFlags.Public | BindingFlags.Static, null, CallingConventions.Any, new[] { typeof(string) }, null);
            if (parse == null) throw new NotSupportedException($"{typeof(TKey)} as TKey in IDictionary<TKey, TValue> is not supported.");
            while (enumerator.MoveNext())
            {
                var element = (KeyValuePair<string?, TValue>)enumerator.Current;
                instance.Add((TKey)parse.Invoke(null, new[] { element.Key }), element.Value);
            }
            return instance;
        }

        public override void Write(Utf8JsonWriter writer, IDictionary<TKey, TValue> value, JsonSerializerOptions options)
        {
            var convertedDictionary = new Dictionary<string?, TValue>(value.Count);
            foreach (var (k, v) in value) convertedDictionary[k?.ToString()] = v;
            JsonSerializer.Serialize(writer, convertedDictionary, options);
            convertedDictionary.Clear();
        }
    }

    sealed class JsonNonStringKeyDictionaryConverterFactory : JsonConverterFactory
    {
        public override bool CanConvert(Type typeToConvert)
        {
            if (!typeToConvert.IsGenericType) return false;
            if (typeToConvert.GenericTypeArguments[0] == typeof(string)) return false;
            return typeToConvert.GetInterface("IDictionary") != null;
        }

        public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options)
        {
            var converterType = typeof(JsonNonStringKeyDictionaryConverter<,>)
                .MakeGenericType(typeToConvert.GenericTypeArguments[0], typeToConvert.GenericTypeArguments[1]);
            var converter = (JsonConverter)Activator.CreateInstance(
                converterType,
                BindingFlags.Instance | BindingFlags.Public,
                null,
                null,
                CultureInfo.CurrentCulture);
            return converter;
        }
    }
}

Test:

class Entity
{
    public string Value { get; set; }
}
class TestClass
{
    public Dictionary<int, Entity> IntKey { get; set; }
    public Dictionary<float, Entity> FloatKey { get; set; }
    public Dictionary<double, Entity> DoubleKey { get; set; }
    public Dictionary<DateTime, Entity> DateTimeKey { get; set; }
    public Dictionary<string, Entity> StringKey { get; set; }
}
class Program
{
    static void Main(string[] args)
    {
        var options = new JsonSerializerOptions();
        options.Converters.Add(new JsonNonStringKeyDictionaryConverterFactory());
        var x = new TestClass
        {
            IntKey = new Dictionary<int, Entity> { [1] = new Entity { Value = "test" } },
            FloatKey = new Dictionary<float, Entity> { [1.3f] = new Entity { Value = "test" } },
            DoubleKey = new Dictionary<double, Entity> { [1.35] = new Entity { Value = "test" } },
            DateTimeKey = new Dictionary<DateTime, Entity> { [DateTime.Now] = new Entity { Value = "test" } },
            StringKey = new Dictionary<string, Entity> { ["test"] = new Entity { Value = "test" } }
        };

        var value = JsonSerializer.Serialize(x, options);
        Console.WriteLine(value);
        var obj = JsonSerializer.Deserialize<TestClass>(value, options);
        Console.WriteLine(JsonSerializer.Serialize(obj, options));
    }
}

Result:

{"IntKey":{"1":{"Value":"test"}},"FloatKey":{"1.3":{"Value":"test"}},"DoubleKey":{"1.35":{"Value":"test"}},"DateTimeKey":{"8/25/2019 6:47:48 PM":{"Value":"test"}},"StringKey":{"test":{"Value":"test"}}}
{"IntKey":{"1":{"Value":"test"}},"FloatKey":{"1.3":{"Value":"test"}},"DoubleKey":{"1.35":{"Value":"test"}},"DateTimeKey":{"8/25/2019 6:47:48 PM":{"Value":"test"}},"StringKey":{"test":{"Value":"test"}}}

However it still cannot serialize a nested Dictionary such as Dictionary<int, Dictionary<int, int>> because System.Text.Json doesn't accept the inner type Dictionary<int, int>. I think it's a bug.

[steveharter]

However it still cannot serialize a nested Dictionary such as Dictionary> because System.Text.Json doesn't accept the inner type Dictionary. I think it's a bug.

Supporting only <string, x> is by design due to scope-cutting in order to ship for v3.0. The 3.0 release is intended to be a minimal viable product with the most common scenarios supported.

[hez2010]

@steveharter At least you shouldn't throw a notsupportexception when there's a useable converter.

[thorgeirk11]

Are there any plans to support this in .net core 3.1?

[buraktamturk]

For the newcomers, the temporary solution is to revert back to Newtonsoft.Json.

1. Add package reference to Microsoft.AspNetCore.Mvc.NewtonsoftJson.
2. Add .AddNewtonsoftJson() just after .AddControllers() / .AddMvc() or any other combination.

[steveharter]

@steveharter At least you shouldn't throw a notsupportexception when there's a useable converter.

Yes that is a fair point. Perhaps we can remove this restriction for 3.1. cc @layomia

Also just to clarify that today dictionary elements are serialized like properties which is possible because the key is a string. Supporting non-string keys means elements will be serialized as a KeyValuePair.

[unruledboy]

Oh boy, had this problem right after I upgraded to 3.0.

Had to install the newton package with AddNewtonsoftJson.

[ahsonkhan]

From @israellot in https://github.com/dotnet/corefx/issues/41345

var dictionary = new Dictionary<int, int>()
            {
                [0] = 1
            };

 var serialized = System.Text.Json.JsonSerializer.Serialize(dictionary);

This simple serialization is handled well by the former default Newtonsoft json library by serializing the int key as string. On System.Text.Json is throws a not supported exception.

[ahsonkhan]

@israellot, @unruledboy, and others on the thread, can you provide details on why your object model requires Dictionaries with integer keys in your scenarios and why changing it to be Dictionary<string, TValue> wouldn't work? I would love to see some usages for gathering requirements and to help motivate the fix.

They would get serialized as strings anyway, so I don't understand in what scenarios you'd want to your underlying dictionary to have int32 keys instead.

[israellot]

@ahsonkhan I believe the key motivation is compatibility.
The previous default serializer was Newtonsoft, so users may have written entire models relying on its ability to serialize and deserialize arbitrary classes. Migrating from 2.X to 3.0 will now cause a silent breaking change since we'll only know the incompatibility at runtime.

I believe many scenarios involve using json just as transport across the wire, and in this case, the domain model might be Dictionary. Your suggestion boils down to creating a separate DTO object Dictionary and converting between the two, seems rather inefficient since now we need to allocate another object just to be compliant with the serializer.
Looking strictly to the serializer, restricting dictionary to having string keys is logical as that's the only possible json representation. But considering that the serializer plays a role in applications I believe it's best to remove as much friction as possible.

[Kieranties]

I've ran into this issue with a small program I'm writing where parts of a version label are provided through a json file.
The label parts have a key that specifies the index where the label part may be inserted. This means the keys are numeric values, e.g

{
  "parts" : {
    "1" : "alpha",
    "3" : "beta"
  }
}

Using Newtonsoft, the json can be deserialized without issue to a Dictionary<int, string>. After converting to System.Text.Json serialization failed.

I've resolved this issue by creating my own DictionaryConverter, and DictionaryConverter.
I also created a simple converter that allows integers to be deserialized from a string

These are then registered through the serializer options: https://github.com/Kieranties/SimpleVersion/blob/master/src/SimpleVersion.Core/Serialization/Serializer.cs#L22

These changes allow the keys of a dictionary to be deserialized instead of directly read as a string. This further opens up support for keys to be arbitrary types that could have their own converters registered for serialization (e.g. enums/type/ types that may be serialized as unique keys etc)

I've not formally tested things but within the current development this seems to have resolved the issue.

[steveharter]

Setting milestone for 3.1 to remove any restrictions that prevents a custom converter from being created that can handle any TKey for Dictionary<TKey,TValue>.

[steveharter]

Update: I've added samples that work with 3.0. I didn't notice any issues such as with nested dictionaries as reported above.

There are two JSON formats being used in the samples:

• JSON object with string properties: {"1":"val1","2":"val2"} even though TKey is not a string.
  
  
  
  • These rely on a TryParse method on the corresponding supported key type. For v3.0 it is not feasible to provide support for a generalized TKey since TryParse methods can be different for any TKey and because there are culture settings that need to be provided (typically Invariant). So the samples below are for a single TKey type.
  
  
  • Sample Dictionary<int, string>. This is a simple example for just TValue == string.
  
  
  • Sample Dictionary<Guid, TValue>. This can handle any TValue.
  
  
  • Sample Dictionary<TKey, TValue> where TKey is Enum. For Enums; this can handle any TValue.
  
  
• JSON array with KeyValuePair entries: [{"Key":1,"Value":"val1"},{"Key":2,"Value":"val2"}]
  
  
  
  • Sample Dictionary<int, string>. This is a simple example for just TValue == string.
  
  
  • Sample Dictionary<TKey, TValue> . This can handle any TKey and TValue because the types are natively represented in JSON and don't require TryParse.
  
  

If these samples are satisfactory, I will change this issue to 5.0 in order to discuss whether we provide built-in support that don't require custom converters.

[steveharter]

Setting milestone to 5.0 for consideration (what if any of the above examples should work by default).

[RandomGHUser]

Deserialization also seems to map generic object types to JsonDocument rather than their normal (primitive?) types.

Example:

string test = "[{\"id\":86,\"name\":\"test\"}]";
var SystemTextJson = System.Text.Json.JsonSerializer.Deserialize<List<Dictionary<string, object>>>(test);
var NewtonSoftJson = Newtonsoft.Json.JsonConvert.DeserializeObject<List<Dictionary<string, object>>>(test);

SystemTextJson[0]["id"] shows as: ValueKind = Number : "86"
NewtonSoftJson[0]["id"] shows as: 86

[roguecode]

@steveharter An enum keyed dictionary with that converter serializes as:
{ "Stuff": [ { "Key": 1, "Value": "String" }, { "Key": 3, "Value": "String" }, { "Key": 2, "Value": "String" } ] }

While JSON.NET gives what I'd assume most people would expect:
{ "Stuff": { "Item1": "String", "Item2": "String", "Item3": "String" }

[steveharter]

Deserialization also seems to map generic object types to JsonDocument rather than their normal (primitive?) types.

Yes that is by design. See https://github.com/dotnet/corefx/issues/38713

[steveharter]

@steveharter An enum keyed dictionary with that converter serializes as:
{ "Stuff": [ { "Key": 1, "Value": "String" }, { "Key": 3, "Value": "String" }, { "Key": 2, "Value": "String" } ] }
While JSON.NET gives what I'd assume most people would expect:
{ "Stuff": { "Item1": "String", "Item2": "String", "Item3": "String" }

I assume you're using the "Sample Dictionary"? If so, yes that used KeyValuePair which has "Key" and "Value" properties. I did not provide a sample for TKey-based enum dictionaries that serialize as property names, but I will work on adding that.

[roguecode]

Yep, that one. And ah ok, I thought you had meant that as a generic dictionary serializer.
Interested to see your new sample when available, as the one we're currently using doesn't seem as fast as I'd want it to be.

[steveharter]

@roguecode here's a Enum sample for Dictionary<TKey, TValue> where TKey is an Enum and uses the "property" JSON syntax instead of KeyValuePair. I also updated the list of samples above to include this new sample.

[ts46235]

Hello, I have something similar but different and I wonder if you can point me to where else to look.

Newtonsoft.Json: 12.0.2
Microsoft.AspNetCore.Mvc.NewtonsoftJson: 3.0.0

Startup running as 3.0:

                .AddMvc(mvcOptions => mvcOptions.EnableEndpointRouting = false)
                .SetCompatibilityVersion(CompatibilityVersion.Version_2_2)
                .AddNewtonsoftJson();

If i have this problemDetails object:

// problemDetails has 2 extension items
{Microsoft.AspNetCore.Mvc.ValidationProblemDetails}
    Detail: "Please refer to the 'errors' property for additional details."
    Errors: Count = 1
    Extensions: Count = 2
    Instance: "/api/test/complex-binding-from-query"
    Status: 400
    Title: "One or more validation errors occurred."
    Type: "https://tools.ietf.org/html/rfc7231#section-6.5.1"

In 2.2, executing Newtonsoft.Json.JsonConvert.SerializeObject(problemDetails) returns

{"errors":{"Int":["The value 'a' is not valid for Int."]},"type":"https://tools.ietf.org/html/rfc7231#section-6.5.1","title":"One or more validation errors occurred.","status":400,"detail":"Please refer to the 'errors' property for additional details.","instance":"/api/test/complex-binding-from-query","traceId":"0HLQQ40AFBJNG","correlationId":"0HLQQ40AFBJNG"}

In 3.0 it returns:

{"Errors":{"param":["The value 'a' is not valid."]},"Type":"https://tools.ietf.org/html/rfc7231#section-6.5.1","Title":"One or more validation errors occurred.","Status":400,"Detail":"Please refer to the 'errors' property for additional details.","Instance":"/bad-request","Extensions":{"traceId":"|d0d40f80-48b6c9184401b0e1.","correlationId":"0HLQR10NSMRGD:00000009"}}

The 3.0 version serialized string includes the IDictionary property name, Extensions, and we can correctly deserialize that string in 3.0. You can see that this property name is left out of the 2.x version.

The problem is that the 3.0 serialization that occurs when the response is returned from a filter using a BadRequestObjectResult, from the code below:

public sealed class ProblemDetailsResultFilterAttribute : Attribute, IAlwaysRunResultFilter
{
        public void OnResultExecuting(ResultExecutingContext context)
        {
             context.Result = new BadRequestObjectResult(problemDetails);
        }
}

..., the response content that gets returned is the same form as the 2.2 version (the Extensions property name is excluded), which causes the Extensions property to deserialize as an empty collection (using Newtonsoft.Json.JsonConvert.DeserializeObject<ValidationProblemDetails>())

Somehow this serialization is not using the same serialization as the Newtonsoft library that we are trying to deserialize with.

Thanks for the consideration!

[layomia]

I have something similar but different

@ts46235 could you please open a new issue for this since this is different from the current issue?

[pranavkm]

@ts46235 I responded to your question in the other issue you opened here - https://github.com/aspnet/AspNetCore/issues/16618. Marking the conversation here as off topic.

[skironDotNet]

updated to Core 3.1 and still not fixed

[richardtallent]

I just upgraded to 3.1 and got hit with this. Back to JSON.NET I go... (I use GUID keys)

[andrew-vdb]

Dot net core 3.1
Dictionary also not working, even the object in the key is actually string

[jeffputz]

I too just hit this, and what a scary silent limitation it is, since as others have pointed out you won't see this at compile time. In my case, I want to serialize Dictionary<int, List<string>>, which doesn't strike me as particularly exotic.

[bklooste]

They should fix it but I see this time and time again even with the old formatter , binary formatter early newtsoft , dictionaries in dictionaries , dictionaries with interfaces . They should fix it but if you dont want trouble people really shouldn't put complex objects like Dictionaries in serialization contracts your asking for trouble - newtsoft has spoiled people . Look at all the public properties on Dictionary count etc you are relying on something custom in the serializer to map this.

Unfortunate there is not a simple type for this in C# for property names so Dictionary is forced. So im just sad..

[cypressious]

Here's a workaround but by no means a complete solution:

   [JsonConverter(typeof(DictionaryConverter))]
   public Dictionary<string, object> ExtraProperties { get; set; } = new Dictionary<string, object>();

public class DictionaryConverter : JsonConverter<Dictionary<string, object>>
{
    public override Dictionary<string, object> Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        var dictionary = JsonSerializer.Deserialize<Dictionary<string, object>>(ref reader, options);

        foreach (string key in dictionary.Keys.ToList())
        {
            if (dictionary[key] is JsonElement je)
            {
                dictionary[key] = Unwrap(je);
            }
        }

        return dictionary;
    }

    public override void Write(Utf8JsonWriter writer, Dictionary<string, object> value, JsonSerializerOptions options)
        => JsonSerializer.Serialize(writer, value, options);

    private static object Unwrap(JsonElement je)
    {
        return je.ValueKind switch
        {
            JsonValueKind.String => je.ToString(),
            JsonValueKind.Number => je.TryGetInt64(out var l) ? l : je.GetDouble(),
            JsonValueKind.True => true,
            JsonValueKind.False => false,
            JsonValueKind.Array => je.EnumerateArray().Select(Unwrap).ToList(),
            JsonValueKind.Object => je.EnumerateObject().ToDictionary(x => x.Name, x => Unwrap(x.Value)),
            _ => null
        };
    }

[am11]

Perhaps support for (some of the) common types serializers could be added OOTB, e.g. this list, and also if one of these types .IsAssignableFrom(systemDotObjectInstance.GetType()) to support Dictionary<object, V>.

[Jozkee]

Here's the proposal for adding support to non-string TKey types in dictionaries.
https://github.com/dotnet/runtime/pull/32676

Please let us know any thoughts or concerns.

[ahsonkhan]

Specifically, please provide feedback on the set of types @Jozkee plans to support for dictionary keys, especially if you need other types to be supported (basically, all the built-in primitive numeric types, enums, and a few others):
https://github.com/dotnet/runtime/blob/a5d96c0e280b56412d4614848f5ee3b1e0d7f216/src/libraries/System.Text.Json/docs/KeyConverter_spec.md#keyconverter

Dictionary also not working, even the object in the key is actually string

@andrew-vdb, supporting serialization of arbitrary object keys will likely remain unsupported. However, if the runtime type of the object keys is one of the "newly supported" types, then serialization will work for that once the feature is done. Deserialization, however, will remain as boxed JsonElement (until this related issue for opting into the different behavior is addressed): https://github.com/dotnet/runtime/issues/29960

[jeffputz]

@Jozkee what are the types enabled for TValue? Presumably anything you can currently serialize as a standalone object?

[Jozkee]

Presumably anything you can currently serialize as a standalone object?

Yes.

[vladislav55]

https://docs.microsoft.com/en-gb/dotnet/standard/serialization/system-text-json-converters-how-to#support-dictionary-with-non-string-key

[kstreichergb]

Deserialization also seems to map generic object types to JsonDocument rather than their normal (primitive?) types.

Example:

string test = "[{\"id\":86,\"name\":\"test\"}]";
var SystemTextJson = System.Text.Json.JsonSerializer.Deserialize<List<Dictionary<string, object>>>(test);
var NewtonSoftJson = Newtonsoft.Json.JsonConvert.DeserializeObject<List<Dictionary<string, object>>>(test);

SystemTextJson[0]["id"] shows as: ValueKind = Number : "86"
NewtonSoftJson[0]["id"] shows as: 86

Of all the mentioned issues, this bothers me the most. List or T[] or Dictionary should deserialize properly for any type that can be mapped directly from the few types Json has to CLR types.

The 3.0 release is intended to be a minimal viable product with the most common scenarios supported

I wonder how List> isn't one of the most common scenarios:

[// N objects
{"a":4},
{"b","Bla"},
]

As this does also deserialize into System.Text.JsonElement, where I would expect double (Number) and string (String)

[AirEssY]

I've ran into this issue with a small program I'm writing where parts of a version label are provided through a json file.
The label parts have a key that specifies the index where the label part may be inserted. This means the keys are numeric values, e.g

{
  "parts" : {
    "1" : "alpha",
    "3" : "beta"
  }
}

Using Newtonsoft, the json can be deserialized without issue to a Dictionary<int, string>. After converting to System.Text.Json serialization failed.

I've resolved this issue by creating my own DictionaryConverter, and DictionaryConverter.
I also created a simple converter that allows integers to be deserialized from a string

These are then registered through the serializer options: https://github.com/Kieranties/SimpleVersion/blob/feature/netcore3/src/SimpleVersion.Core/Serialization/Serializer.cs#L20

These changes allow the keys of a dictionary to be deserialized instead of directly read as a string. This further opens up support for keys to be arbitrary types that could have their own converters registered for serialization (e.g. enums/type/ types that may be serialized as unique keys etc)

I've not formally tested things but within the current development this seems to have resolved the issue.

hey @Kieranties , the github links 404 for me

[Kieranties]

hey @Kieranties , the github links 404 for me

@AirEssY I've corrected the links in my original comment, thinks are now in master at: https://github.com/Kieranties/SimpleVersion/tree/master/src/SimpleVersion.Core/Serialization

[derekjwilliams]

If the Dictionary is equivalent to a JavaScript Map, then any (JS type represented in C#) should be acceptable,

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map

A Map's keys can be any value (including functions, objects, or any primitive).

An example of the standard approach to deserialize a Map in JS is:

const countries = new Map(JSON.parse('[[1,"Bahamas (the)"],[2,"Bolivia (Plurinational State of)"]]'))

console.log(countries)

Which produces:

Map(2) {
  1 => 'Bahamas (the)',
  2 => 'Bolivia (Plurinational State of)'
}

TL;DR: restricting keys to strings does not play well with JS

[onionhammer]

@Jozkee so is this coming in .NET 5 only or will it be go into 3.*?

[Jozkee]

@onionhammer .NET 5.0, you can also try out the feature in the next preview (5.0 preview8).
There are no plans of porting this into 3.x.

[verloka]

Solution for asp net core 3.x:

var dic1 = new Dictionary<TKey, TValue>(); 
return Json(new { dic1 }); // does not work

var dic2 = from i in new Dictionary<TKey, TValue>() select new { i.Key, i.Value }
return Json(new { dic2 });  //works prety well

[onionhammer]

@verloka that is not the desired output

[layomia]

@Jozkee so is this coming in .NET 5 only or will it be go into 3.*?

This won't be backported to 3.x but you can add use the System.Text.Json NuGet package in your project to get all the new features in .NET 5.