Iotedge: IoT Edge - AMQP messages are not received, and error in sending the messages

[keshava-hm]

I'm trying to send/receive messages to/from another iot edge module.

Following is the code:

var amqpSetting = new AmqpTransportSettings(TransportType.Amqp_Tcp_Only);
ITransportSettings[] settings = { amqpSetting };

// Open a connection to the Edge runtime
this.IoTHubModuleClient = await ModuleClient.CreateFromEnvironmentAsync(settings);

await this.IoTHubModuleClient.OpenAsync();

For receive:

await this.IoTHubModuleClient.SetInputMessageHandlerAsync("mainInput", this.ReceiveMessage, this.IoTHubModuleClient);

For send:

await moduleClient.SendEventAsync("mainOutput", upstreamMessage);
ReceiveMessage is not being invoked even though the other module is sending the message, and I get following exception after some time for SendEventAsync

Microsoft.Azure.Devices.Client.Exceptions.IotHubCommunicationException: The operation did not complete within the allocated time 00:01:00 for object message. ---> Microsoft.Azure.Devices.Client.Exceptions.IotH
ubCommunicationException: The operation did not complete within the allocated time 00:01:00 for object message. ---> System.TimeoutException: The operation did not complete within the allocate
d time 00:01:00 for object message.
at Microsoft.Azure.Amqp.AsyncResult.EndTAsyncResult
at Microsoft.Azure.Amqp.SendingAmqpLink.EndSendMessage(IAsyncResult result)
at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTSendingLink.SendAmqpMessageAsync(AmqpMessage amqpMessage, TimeSpan timeout)
--- End of inner exception stack trace ---
--- End of inner exception stack trace ---
at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTSendingLink.SendAmqpMessageAsync(AmqpMessage amqpMessage, TimeSpan timeout)
at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpIoTSendingLink.SendMessageAsync(Message message, TimeSpan timeout)
at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.SendMessageAsync(Message message, TimeSpan timeout)
at Microsoft.Azure.Devices.Client.Transport.AmqpIoT.AmqpUnit.SendEventAsync(Message message, TimeSpan timeout)
at Microsoft.Azure.Devices.Client.Transport.Amqp.AmqpTransportHandler.SendEventAsync(Message message, CancellationToken cancellationToken)
at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.<>c__DisplayClass22_0.<b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.Azure.Devices.Client.Transport.ErrorDelegatingHandler.ExecuteWithErrorHandlingAsyncT

But this message is being received in the other IoT edge module.

Can someone help me with this?

Thanks.

[darobs]

Hello @keshava-hm,

This may be off-base, but is your message handler returning MessageResponse.Completed when its done?

[keshava-hm]

Hello @keshava-hm,

This may be off-base, but is your message handler returning MessageResponse.Completed when its done?

yes, it does.

[darobs]

Darn,

Are you able to provide edgeHub logs? It would also be useful to get version information:

Device Information

• Host OS [e.g. Ubuntu 16.04, Ubuntu 18.04, Windows IoT Core]:
• Architecture [e.g. amd64, arm32, arm64]:
• Container OS [e.g. Linux containers, Windows containers]:

Runtime Versions

• iotedged [run iotedge version]:
• Edge Agent [image tag (e.g. 1.0.0)]:
• Edge Hub [image tag (e.g. 1.0.0)]:
• Docker/Moby [run docker version]:

[keshava-hm]

Darn,

Are you able to provide edgeHub logs? It would also be useful to get version information:

Device Information

• Host OS [e.g. Ubuntu 16.04, Ubuntu 18.04, Windows IoT Core]:
• Architecture [e.g. amd64, arm32, arm64]:
• Container OS [e.g. Linux containers, Windows containers]:

Runtime Versions

• iotedged [run iotedge version]:
• Edge Agent [image tag (e.g. 1.0.0)]:
• Edge Hub [image tag (e.g. 1.0.0)]:
• Docker/Moby [run docker version]:

Hi @darobs , I've updated the root cause for the issue and few questions in other thread. Kindly help me on that. I'm closing this issue now.

[asergaz]

@keshava-hm please reopen the issue here so that we continue the troubleshooting. Thanks!

[keshava-hm]

@asergaz As I explained in the other thread, I face issues if I create a new edge module as per the documentation and try to receive Data from simulated temperature sensor. Can you please check if this issue is happening in your build system as well? The issue is happening only with Visual studio and not with visual studio code. Thank you.

[asergaz]

@keshava-hm please can you provide the info requested by @darobs as well? Thanks!

[keshava-hm]

@asergaz oops. I missed that post. Sorry.

Here's the info:
Host OS: Win10
Arch : amd64
Windows containers

Will update the other informations in sometime..
Thanks.

[keshava-hm]

Additional info requested:
iotedge version
iotedge 1.0.8 (208b2204fd30e856d00b280112422130c104b9f0)

edge agent : mcr.microsoft.com/azureiotedge-agent:1.0
edge hub: mcr.microsoft.com/azureiotedge-hub:1.0
docker version
Client:
Version: 3.0.5
API version: 1.40
Go version: go1.12.1
Git commit: ba9934d4
Built: Thu Apr 18 22:01:41 2019
OS/Arch: windows/amd64
Experimental: false

Also request you to check https://github.com/MicrosoftDocs/azure-docs/issues/40855 since that was the easy way to reproduce and it's the root cause I believe.

Thanks.
--Keshava

[asergaz]

Important detail, @keshava-hm is using Azure VM with IoT Edge. Are there restrictions using AMQP on an Azure VM known?

[keshava-hm]

@asergaz hmm yes. But tried on real edge device with win10 1809 as well. Its the same result.
--Keshava

[keshava-hm]

related threads: https://github.com/Azure/azure-iot-sdk-csharp/issues/1108
https://github.com/MicrosoftDocs/azure-docs/issues/40855

[philipktlin]

1. try to update sender code to use this SendEventAsync method without output name (https://docs.microsoft.com/en-us/dotnet/api/microsoft.azure.devices.client.moduleclient.sendeventasync?view=azure-dotnet#Microsoft_Azure_Devices_Client_ModuleClient_SendEventAsync_Microsoft_Azure_Devices_Client_Message_).
2. try to update receiver code to use this SetMessageHandlerAsync (https://docs.microsoft.com/en-us/dotnet/api/microsoft.azure.devices.client.moduleclient.setmessagehandlerasync?view=azure-dotnet#Microsoft_Azure_Devices_Client_ModuleClient_SetMessageHandlerAsync_Microsoft_Azure_Devices_Client_MessageHandler_System_Object_) instead of SetInputMessageHandlerAsync method.
3. Share the route config of EdgeHub.

[keshava-hm]

1. try to update sender code to use this SendEventAsync method without output name (https://docs.microsoft.com/en-us/dotnet/api/microsoft.azure.devices.client.moduleclient.sendeventasync?view=azure-dotnet#Microsoft_Azure_Devices_Client_ModuleClient_SendEventAsync_Microsoft_Azure_Devices_Client_Message_).
2. try to update receiver code to use this SetMessageHandlerAsync (https://docs.microsoft.com/en-us/dotnet/api/microsoft.azure.devices.client.moduleclient.setmessagehandlerasync?view=azure-dotnet#Microsoft_Azure_Devices_Client_ModuleClient_SetMessageHandlerAsync_Microsoft_Azure_Devices_Client_MessageHandler_System_Object_) instead of SetInputMessageHandlerAsync method.
3. Share the route config of EdgeHub.

Hi @philipktlin , I had opened another thread which explains the actual issue which caused this issue in my project, and the workaround I used. https://github.com/MicrosoftDocs/azure-docs/issues/40855
So according to this thread, the issue was introduced when upgrading Microsoft.Azure.Devices.Client 1.21.0 to 1.21.1.

Thanks
--Keshava G N

[keshava-hm]

In my opinion, this issue can be closed and the other actual issue (https://github.com/MicrosoftDocs/azure-docs/issues/40855) can be investigated...
The same issue is reported in Linux in https://github.com/Azure/azure-iot-sdk-csharp/issues/1108 as well

[philipktlin]

Thanks for your reply.

[keshava-hm]

Looks like this issue is solved in 1.21.2
https://github.com/Azure/azure-iot-sdk-csharp/issues/1108