Azure-iot-sdk-csharp: InvalidOperationException thrown when SendEventBatchAsync is called on the DeviceClient.

[namalu]

• OS, version, SKU and CPU architecture used: macOS Mojave 10.14.6 x64
• Application's .NET Target Framework : xamarin
• Device: iPhone & Android Phones
• SDK version used: 1.21.1

Description of the issue:

InvalidOperationException thrown when SendEventBatchAsync is called on the DeviceClient.

It appears that on line 79 of the AmqpIoTSendingLink class, DataBody is null on the AmqpMessage. This is because on line 58 of the AmqpIoTMessageConverter class the AmqpMessage is initialized with a Stream. The BodyStream property should be used instead of DataBody.

Code sample exhibiting the issue:

```C#
var message = new Message(Encoding.ASCII.GetBytes(jsonString));
_queue.Enqueue(message);

if (_queue.Count >= 30)
{
var messages = _queue.ToArray();
try
{
await _deviceClient.SendEventBatchAsync(messages);
}
catch (Exception e)
{
// Exception thrown here
System.Diagnostics.Debug.WriteLine(e);
}
}
```

Console log of the issue:

System.InvalidOperationException: Operation is not valid due to the current state of the object.
at Microsoft.Azure.Amqp.AmqpMessage.get_DataBody () [0x00000] in <9cea9f83a18d449284cb86dea42f9d95>:0
at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTSendingLink.SendMessagesAsync (System.Collections.Generic.IEnumerable1[T] messages, System.TimeSpan timeout) [0x00067] in <8ffa96f34a5347a3a51154700c91224d>:0 at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.SendMessagesAsync (System.Collections.Generic.IEnumerable1[T] messages, System.TimeSpan timeout) [0x0011b] in <8ffa96f34a5347a3a51154700c91224d>:0
at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.SendEventsAsync (System.Collections.Generic.IEnumerable1[T] messages, System.TimeSpan timeout) [0x0009f] in <8ffa96f34a5347a3a51154700c91224d>:0 at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.SendEventAsync (System.Collections.Generic.IEnumerable1[T] messages, System.Threading.CancellationToken cancellationToken) [0x000ae] in <8ffa96f34a5347a3a51154700c91224d>:0
at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler+<>c__DisplayClass22_0.b__0 () [0x00071] in <8ffa96f34a5347a3a51154700c91224d>:0
at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsync[T] (System.Func1[TResult] asyncOperation) [0x0014c] in <8ffa96f34a5347a3a51154700c91224d>:0 at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler+<>c__DisplayClass15_0.<SendEventAsync>b__0 () [0x00135] in <8ffa96f34a5347a3a51154700c91224d>:0 at Microsoft.Azure.Devices.Client.Transport.RetryDelegatingHandler.SendEventAsync (System.Collections.Generic.IEnumerable1[T] messages, System.Threading.CancellationToken cancellationToken) [0x00101] in <8ffa96f34a5347a3a51154700c91224d>:0
at Microsoft.Azure.Devices.Client.InternalClient.SendEventBatchAsync (System.Collections.Generic.IEnumerable`1[T] messages) [0x00092] in <8ffa96f34a5347a3a51154700c91224d>:0

[CIPop]

Thanks for reporting @namalu ! We are aware of several Xamarin issues with our SDKs and we are tracking part of a larger project.
I do remember we have had fixes in our latest release (today): please try again and let us know .

[ntrpnr]

I have the same issue on .NET Standard 2.0 and AMQP TCP Only.

We had to fallback to foreach SendEventAsync.

[mnmadscience]

I am also seeing this on .NET Standard 2.0 and AMQP TCP. This isn't Xamarin specific, I think.

My codebase is fairly large, so it would not be easy to provide the whole thing. However I'd be glad to provide code snippets. I can also perform experiments/answer questions as needed.

[mnmadscience]

Confirmed that ntrpnr's workaround works in my code as well! Just do a foreach SendEventAsync.

[OliverNET]

Same issue here with an ASP.NET Core 3 app.

Works with Microsoft.Azure.Devices.Client with Version 1.20.3
Fails with all newer nuget packages.

As far as i understand, the issue with fallback to foreach & SendEventAsync() is that you get charged/throttled differently.

E.g. 10 messages with 200 bytes each:
SendEventAsync() => 10 messages charged
SendEventBatchAsync() => 1 message charged

[namalu]

Thanks for reporting @namalu ! We are aware of several Xamarin issues with our SDKs and we are tracking part of a larger project.
I do remember we have had fixes in our latest release (today): please try again and let us know .

Definitely not Xamarin specific and this issue still persists in the latest Nuget.

[timtay-microsoft]

@namalu if you are willing to provide a PR for this issue, I'd be happy to merge it in for you.

[ntrpnr]

Any update on this issue? SendEventBatchAsync was working in a recent version and is present in the official documentation but does not simply work, at least not over AMQP.

As previously noted, this is not limited to Xamarin.

[Peter-B-]

I took a look into that problem today. I am a bit confused on how it should work, but the issue is the following:

Starting from line 75 of AmqpIoTSendingLink.SendMessagesAsync there is the following statement:

using (AmqpMessage amqpMessage = AmqpIoTMessageConverter.MessageToAmqpMessage(message))
{
    var data = new Data()
    {
        Value = amqpMessage.DataBody
    };
    messageList.Add(data);
}

MessageToAmqpMessage(message) returns a AmqpBodyStreamMessage object which does not implement DataBody. It would implement a BodyStream property instead.

I do not understand the implication of this or what the correct fix might be, but maybe this can help you to locate the issue.

I tried to write a unit test for this scenario, but it feels like AmqpIoTSendingLink might require some refactoring first.

[schungx]

I can't understand why such a core feature can be broken without anybody in Microsoft's QC department spotting this BEFORE the release got out?

[davilu]

Released in https://github.com/Azure/azure-iot-sdk-csharp/releases/tag/2019-12-17

[az-iot-builder-01]

@CIPop, @ntrpnr, @jbmadscience, @OliverNET, @namalu, @Peter-B-, @schungx, @davilu, thank you for your contribution to our open-sourced project! Please help us improve by filling out this 2-minute customer satisfaction survey

[OliverNET]

Thank you for fixing this bug.
It's working now 👍

[Bjarkes]

Will this fix be merged into the preview branch any time soon? As we are using the DeviceStream preview feature we are forced to use only the 1.29.0-preview-* versions, which does not include the fix yet.

[prmathur-microsoft]

Preview branch is updated once every month. It should be apart of next preview release.