Azure-iot-sdk-csharp: Exception thrown when calling RegistryManager.GetTwinAsync()

[andyce1010]

OS and version used: Windows 10 Pro
SDK version used:
Microsoft.Azure.Devices 1.4.1
.NET Core 2.0

Description of the issue:
Any calls to RegistryManager.GetTwinAsync("deviceId") seem to result in a JsonSerializationException being thrown.
This seems to have suddenly started happening today, so I not sure what has changed.

I've have created a simple .NET core console app to reproduce the issue (see code below), and I get the same error.

I also created a full framework console app, and that one works with out issue. So it must be related to .NET Core somehow.

Code sample exhibiting the issue:

    static void Main(string[] args)
    {
        var man = RegistryManager.CreateFromConnectionString("xxxxxxx");

        var twin = man.GetTwinAsync("xxxxx").Result;
    }

Console log of the issue:

System.AggregateException occurred
HResult=0x80131500
Message=One or more errors occurred. (Additional text found in JSON string after finishing deserializing object. Path '', line 1, position 272.)
Source=System.Private.CoreLib
StackTrace:
at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
at ConsoleApp2.Program.Main(String[] args) in C:\dev\ConsoleApp2\Program.cs:line 12

Inner Exception 1:
IotHubException: Additional text found in JSON string after finishing deserializing object. Path '', line 1, position 272.

Inner Exception 2:
JsonSerializationException: Additional text found in JSON string after finishing deserializing object. Path '', line 1, position 272.

[jonasaebersold]

We have the same problem in our project. We have integration tests which fail now but we did not change anything in our source code. If we switch the test project from net core 2.0 to .net 4.6.1, the test works again. There has to be a change in the JSON-telegram, because our code has not changed but from one day to the other we cannot read the device-twin anymore.

[Jaternius]

After investigating the issue a bit, we have found the source of the issue:

https://github.com/Azure/azure-iot-sdk-csharp/blob/master/shared/Microsoft.Azure.Devices.Shared/TwinJsonConverter.cs#L89-L92

The problem arises in a JSON telegram change. There is a new property, called capabilities, which is not handled by the TwinJsonConverter. Therefore, it's thinking that the EndObject tag belongs to the telegram itself and ends the parsing.

Example JSON telegram as received by the server:

{
    "deviceId": "azuretest",
    "etag": "AAAAAAAAAAE=",
    "version": 2,
    "properties": {
        "desired": {
            "$metadata": {
                "$lastUpdated": "2017-11-02T12:33:47.6569425Z"
            },
            "$version": 1
        },
        "reported": {
            "$metadata": {
                "$lastUpdated": "2017-11-02T12:33:47.6569425Z"
            },
            "$version": 1
        }
    },
    "capabilities": {
        "iotEdge": false
    }
}

I think because .NET Standard and the full .NET use different serializers, this issue does not arise in the full .NET version. We haven't tested that though.

[jasmineymlo]

This is a known issue. It only affects .Net standard and core clients. We are in the process of fixing this on the service side. Until then a workaround is to call the following before the call to GetTwinAsync:

 JsonConvert.DefaultSettings = () => new JsonSerializerSettings
    {
        CheckAdditionalContent = false
    };

Thanks.

[yfakariya]

Any update for this? This issue looks critical for on-going services.

[jasmineymlo]

The fix will be in this week's deployment. Thanks.

[yfakariya]

@jasmineymlo Although it sounds great, I think the deserialization logic should be forward compatible (setting CheckAdditionalContent = false, for example) to prevent future degradation. Is there any plan to add such preventive logic, or unify twin deserializers?

[jasmineymlo]

@yfakariya Thanks for the suggestion. Would you mind opening another issue to track for this enhancement? I am closing this issue which service has deployed a fix for.

[yfakariya]

@jasmineymlo Sure! I post a reminder issue to https://github.com/Azure/azure-iot-sdk-csharp/issues/341 now.