Autorest: Anonymous types are not getting properly serialized

[keerthivasanm]

The below request gets converted,

var extendedData = new[]
    {                
        new { Name = "foo", Value = 1 },
        new { Name = "bar", Value = 2 },
    };
request.ExtendedData = extendedData;

into following JSON

{"Data":"test","Monitoring":true,"ExtendedData":[{},{}]}

with SafeJsonConvert.SerializeObject. It works as expected with JsonConvert.Serialize method of JSON.NET.

[brjohnstmsft]

SafeJsonConvert.SerializeObject is nearly identical to JsonConvert.SerializeObject except that it doesn't inherit global JsonSerializerSettings from JsonConvert. Are you using custom settings in one case but not the other?

Also, can you provide the Swagger spec as well as the generated code for the type of request?

[keerthivasanm]

I'm using the default auto-generated serialization settings with no modifications for both SafeJsonConvert and JsonConvert.

Part of Swagger spec:

{
  "ServiceRequest": {
    "type": "object",
    "properties": {
      "SessionId": {
        "type": "string"
      },
      "ZipCode": {
        "type": "string"
      },
      "VisitorId": {
        "type": "string"
      },
      "ExtendedData": {
        "type": "object",
        "extendedData": {
          "$ref": "#/definitions/Object"
        }
      }
    }
  },
  "Object": {
    "type": "object",
    "properties": {}
  }
}

Extended data is just a Dictionary<string,object> and in this context, object is an anonymous type with properties Name and Value.

AutoRest generated class:

// Code generated by Microsoft (R) AutoRest Code Generator 0.15.0.0
// Changes may cause incorrect behavior and will be lost if the code is
// regenerated.

namespace SearchApi.Client.AutoRest.Models
{
    using System;
    using System.Linq;
    using System.Collections.Generic;
    using Newtonsoft.Json;
    using Microsoft.Rest;
    using Microsoft.Rest.Serialization;

    public partial class ServiceRequest
    {
        /// <summary>
        /// Initializes a new instance of the ServiceRequest class.
        /// </summary>
        public ServiceRequest() { }

        /// <summary>
        /// Initializes a new instance of the ServiceRequest class.
        /// </summary>
        public ServiceRequest(string sessionId = default(string), string zipCode = default(string), string visitorId = default(string), IDictionary<string, object> extendedData = default(IDictionary<string, object>))
        {
            SessionId = sessionId;
            ZipCode = zipCode;
            VisitorId = visitorId;
            ExtendedData = extendedData;
        }

        /// <summary>
        /// </summary>
        [JsonProperty(PropertyName = "SessionId")]
        public string SessionId { get; set; }

        /// <summary>
        /// </summary>
        [JsonProperty(PropertyName = "ZipCode")]
        public string ZipCode { get; set; }

        /// <summary>
        /// </summary>
        [JsonProperty(PropertyName = "VisitorId")]
        public string VisitorId { get; set; }

        /// <summary>
        /// </summary>
        [JsonProperty(PropertyName = "ExtendedData")]
        public IDictionary<string, object> ExtendedData { get; set; }

    }
}

[brjohnstmsft]

Since the type of ExtendedData is IDictionary<string, object>, how is this even compiling?

var extendedData = new[]
    {                
        new { Name = "foo", Value = 1 },
        new { Name = "bar", Value = 2 },
    };
request.ExtendedData = extendedData;

Also, how are you trying it with JsonConvert? Just by tweaking the generated code?

[keerthivasanm]

I was trying to provide a sample and ended up with a bad one. It should be

var dict = new Dictionary<string, object>();
var extendedData = new[]
    {
           new { Name = "foo", Value = 1 },
           new { Name = "bar", Value = 2 },
    };
dict.Add("someData", extendedData);
request.ExtendedData = extendedData;

Since the project already has a reference to Newtonsoft.Json library, I used JsonConvert from QuickWatch window.

[brjohnstmsft]

I tried this out myself with AutoRest 0.15.0, and here's what I found:

• The generated code for ServiceRequest types ExtendedData as object, not as IDictionary<string, object>.
• Both SafeJsonConvert and JsonConvert serialize the test payload to {"ExtendedData":[{},{}]}.

Here is the Swagger I used:

{
  "swagger": "2.0",
  "info": {
    "title": "TestClient",
    "version": "2016-04-11"
  },
  "consumes": [
    "application/json"
  ],
  "produces": [
    "application/json"
  ],
  "paths": {
    "/values": {
      "get": {
        "operationId": "ListValues",
        "responses": {
          "200": {
            "description": "",
            "schema": {
              "$ref": "#/definitions/ServiceRequest"
            }
          }
        }
      },
      "post": {
        "operationId": "CreateValue",
        "parameters": [
          {
            "name": "payload",
            "in": "body",
            "required": true,
            "schema": {
              "$ref": "#/definitions/ServiceRequest"
            }
          }
        ],
        "responses": {
          "201": {
            "description": "",
            "schema": {
              "$ref": "#/definitions/ServiceRequest"
            }
          }
        }
      }
    }
  },
  "definitions": {
    "ServiceRequest": {
        "type": "object",
        "properties": {
            "SessionId": {
                "type": "string"
            },
            "ZipCode": {
                "type": "string"
            },
            "VisitorId": {
                "type": "string"
            },
            "ExtendedData": {
                "type": "object",
                "extendedData": {
                "$ref": "#/definitions/Object"
                }
            }
        }
    },
    "Object": {
        "type": "object",
        "properties": {}
    }
  }
}

Here's the generated code for ServiceRequest:

// Code generated by Microsoft (R) AutoRest Code Generator 0.15.0.0
// Changes may cause incorrect behavior and will be lost if the code is
// regenerated.

namespace Test.Models
{
    using System;
    using System.Linq;
    using System.Collections.Generic;
    using Newtonsoft.Json;
    using Microsoft.Rest;
    using Microsoft.Rest.Serialization;

    public partial class ServiceRequest
    {
        /// <summary>
        /// Initializes a new instance of the ServiceRequest class.
        /// </summary>
        public ServiceRequest() { }

        /// <summary>
        /// Initializes a new instance of the ServiceRequest class.
        /// </summary>
        public ServiceRequest(string sessionId = default(string), string zipCode = default(string), string visitorId = default(string), object extendedData = default(object))
        {
            SessionId = sessionId;
            ZipCode = zipCode;
            VisitorId = visitorId;
            ExtendedData = extendedData;
        }

        /// <summary>
        /// </summary>
        [JsonProperty(PropertyName = "SessionId")]
        public string SessionId { get; set; }

        /// <summary>
        /// </summary>
        [JsonProperty(PropertyName = "ZipCode")]
        public string ZipCode { get; set; }

        /// <summary>
        /// </summary>
        [JsonProperty(PropertyName = "VisitorId")]
        public string VisitorId { get; set; }

        /// <summary>
        /// </summary>
        [JsonProperty(PropertyName = "ExtendedData")]
        public object ExtendedData { get; set; }

    }
}

Here's the unit test I wrote:

using System;
using Microsoft.Rest;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Test;
using Test.Models;

namespace TestClient.Tests
{
    [TestClass]
    public class UnitTest1
    {
        [TestMethod]
        public void TestMethod1()
        {
            var client = new Test.TestClient(new Uri("http://localhost"));

            var payload = new ServiceRequest()
            {
                ExtendedData = new[]
                {
                    new { Name = "foo", Value = 1 },
                    new { Name = "bar", Value = 2 },
                }
            };

            client.CreateValue(payload);
        }
    }
}

I set a breakpoint in CreateValueWithHttpMessagesAsync and used the debugger to observe request serialization.

I'm using the same version of AutoRest as you, so I'm not sure why what I'm seeing is different, but it confirms how I expect SafeJsonConvert to behave (that is, exactly the same as JsonConvert). That said, in principle there is no reason it shouldn't support anonymous types. I'll poke on it some more and see if it has to do with the serialization settings.

In the meantime, you can work around it by using a dictionary instead:

            var payload = new ServiceRequest()
            {
                ExtendedData = new Dictionary<string, object>()
                {
                    { "foo", 1 },
                    { "bar", 2 }
                }
            };

Also, in the future, please provide a more complete repro for issues that you report. It helps you in the end by speeding up investigation of the issue. Thanks!

[brjohnstmsft]

I've confirmed that this problem is caused by the ReadOnlyJsonContractResolver. Here is a minimal repro:

using Microsoft.Rest.Serialization;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using Newtonsoft.Json;

namespace JsonAnonymousRepro
{
    [TestClass]
    public class UnitTest1
    {
        [TestMethod]
        public void TestMethod1()
        {
            var serializationSettings = new JsonSerializerSettings
            {
                // Uncomment this line to make the test fail.
                ////ContractResolver = new ReadOnlyJsonContractResolver(),
            };

            object o = new { Name = "foo", Value = 1 };

            string json = JsonConvert.SerializeObject(o, serializationSettings);
            Assert.AreEqual(@"{""Name"":""foo"",""Value"":1}", json);
        }
    }
}

I'll investigate further.

[amarzavery]

One needs to specify additionalProperties for it to be a dictionary. Please look at the open api specification over here https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#model-with-mapdictionary-properties . An Object with empty properties defined inside the definitions section does not qualify to become a dictionary.

[keerthivasanm]

@amarzavery: The swagger spec has additionalProperties and AutoRest generated code also recognizes it as Dictionary. The issue is when one of the value is a list of anonymous type.

[amarzavery]

@keerthivasanm - I see.
As you are saying

object is an anonymous type with properties Name and Value .

Do you expect more properties to be present or just Name and Value?
Do Name and Value have predefined types or they can be anything?

The reason for asking these questions is, that when you term it as "anonymous" do you mean "It could by any valid JSON object" and properties name and value were just given here to elaborate your example?

[brjohnstmsft]

Whether the Swagger contains additionalProperties or not is a red herring. The core issue is that ReadOnlyContractResolver prevents anonymously-typed objects from being serialized correctly. This is because properties in anonymous types are automatically "public get, private set", and these properties are specifically ignored by ReadOnlyContractResolver, because that's its job.

@keerthivasanm Let's say you are able to serialize an anonymously-typed object in ServiceRequest.ExtendedData, and then you round-trip that back from the server. How do you expect the object to be typed in the response?

[keerthivasanm]

@brjohnstmsft: Assuming the transport is JSON, it either could be a JObject or JArray. Since it is an anonymous type, the client / server can agree upon what could be sent / expected for a given key. This just gives some flexibility to add additional data without making any changes to the contract.

[brjohnstmsft]

Thanks for the details @keerthivasanm

I've made sure this issue is being triaged by the right people.

FYI @markcowl

[fearthecowboy]

O_o

IIRC, JSON.NET does not serialize anonymous types.

If you want to transport data that is untyped, the additionalProperties feature may help you:

https://github.com/Azure/autorest/blob/master/Documentation/defining-clients-swagger.md#dictionaries

[brjohnstmsft]

@fearthecowboy JSON.NET serializes anonymous types just fine. The reason they don't work with AutoRest-generated clients is that ReadOnlyContractResolver prevents it (because properties of anonymous types are read-only). This is orthogonal to additionalProperties. I think it's fair to just say "AutoRest doesn't support anonymous types; Use a dictionary instead" but AFAIK nobody has actually confirmed that.

[fearthecowboy]

Ah, right... I was thinking of a time I was using anon types with dynamic...

Saying it then : "AutoRest doesn't support anonymous types; Use a dictionary instead". :)

[nrandell]

I've just found this and solved it by created a CustomInitialize that changes the contract resolver.

        partial void CustomInitialize()
        {
            SerializationSettings.ContractResolver = new DefaultContractResolver();
        }

My code is now working!