Mqttnet: Client hangs when reconnecting

Created on 29 May 2019 · 21Comments · Source: chkr1011/MQTTnet

Hello,
I'm facing an issue where when the client tries to reconnect to the broker it hangs.

Application is:

ASP.NET Core 2.2
MQTTNet Version 3.0.2

To test what I'm doing I have a Mosquito MQTT message broker running locally in Docker which I can start (docker start <containerid>) and stop (docker stop <containerid>).

Here is my code:

public async Task ConnectAsync()
        {
            _mqttClient.UseConnectedHandler(e =>
            {
                _logger.LogWarning(EventIds.ConnectionEstablished, "Connection established to broker");
            });

            _mqttClient.UseDisconnectedHandler(async e =>
            {
                _logger.LogWarning(EventIds.Disconnected, "Disconnected from broker");

                if (_isDisconnecting) return;

                await Retry();  //.ConfigureAwait(false);
            });

            await _mqttClient.ConnectAsync(_clientOptions).ConfigureAwait(false);
        }

        private async Task Retry()
        {
            _logger.LogWarning(EventIds.Disconnected, "Retrying in 5 seconds");
            await Task.Delay(TimeSpan.FromSeconds(5));

            try
            {
                await _mqttClient.ConnectAsync(_clientOptions); //.ConfigureAwait(false);
            }
            catch
            {
                _logger.LogError("####### Reconnect failed #######");
            }
        }

When I start the app with the message broker off, it works as expected. The disconnect handler is fired every time it fails to connect, resulting in a 5 second delay and then another attempt at reconnecting. Switching the broker back on results in it reconnecting and goes ahead as normal, sending pings.

The problem comes when it has already established a connection and the broker is stopped. The disconnect handler is fired as expected, but when it tries to connect the application hangs.

Here are the logs:

[17:43:34.501 VRB] MqttNetLogEvent. LogId: null, Message: Stopped sending keep alive packets., Source: MqttClient, ThreadId: 4, Timestamp: 05/28/2019 16:43:34
[17:43:34.505 VRB] MqttNetLogEvent. LogId: null, Message: Disconnecting [Timeout=00:00:10], Source: MqttClient, ThreadId: 4, Timestamp: 05/28/2019 16:43:34
[17:43:34.509 VRB] MqttNetLogEvent. LogId: null, Message: Disconnected from adapter., Source: MqttClient, ThreadId: 4, Timestamp: 05/28/2019 16:43:34
[17:43:34.512 INF] MqttNetLogEvent. LogId: null, Message: Disconnected., Source: MqttClient, ThreadId: 4, Timestamp: 05/28/2019 16:43:34
[17:43:34.515 WRN] Disconnected from broker
[17:43:34.517 WRN] Retrying in 5 seconds
[17:43:39.532 VRB] MqttNetLogEvent. LogId: null, Message: Trying to connect with server 'localhost:1883' (Timeout=00:00:10)., Source: MqttClient, ThreadId: 8, Timestamp: 05/28/2019 16:43:39
[17:43:41.766 ERR] MqttNetLogEvent. LogId: null, Message: Error while connecting with server., Source: MqttClient, ThreadId: 5, Timestamp: 05/28/2019 16:43:41
MQTTnet.Exceptions.MqttCommunicationException: No connection could be made because the target machine actively refused it [::ffff:127.0.0.1]:1883 ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: No connection could be made because the target machine actively refused it [::ffff:127.0.0.1]:1883
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.Socket.DoMultipleAddressConnectCallback(Object result, MultipleAddressConnectAsyncResult context)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Sockets.Socket.DoMultipleAddressConnectCallback(Object result, MultipleAddressConnectAsyncResult context)
   at System.Net.Sockets.Socket.MultipleAddressConnectCallback(IAsyncResult result)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.Socket.<>c.<ConnectAsync>b__274_0(IAsyncResult iar)
--- End of stack trace from previous location where exception was thrown ---
   at MQTTnet.Implementations.MqttTcpChannel.ConnectAsync(CancellationToken cancellationToken) in C:\Users\lbostock\Source\Repos\MQTTnet\Source\MQTTnet\Implementations\MqttTcpChannel.cs:line 67
   at MQTTnet.Adapter.MqttChannelAdapter.<ConnectAsync>b__30_0(CancellationToken t) in C:\Users\lbostock\Source\Repos\MQTTnet\Source\MQTTnet\Adapter\MqttChannelAdapter.cs:line 70
   at MQTTnet.Internal.MqttTaskTimeout.WaitAsync(Func`2 action, TimeSpan timeout, CancellationToken cancellationToken) in C:\Users\lbostock\Source\Repos\MQTTnet\Source\MQTTnet\Internal\MqttTaskTimeout.cs:line 19
   at MQTTnet.Adapter.MqttChannelAdapter.ConnectAsync(TimeSpan timeout, CancellationToken cancellationToken) in C:\Users\lbostock\Source\Repos\MQTTnet\Source\MQTTnet\Adapter\MqttChannelAdapter.cs:line 70
   --- End of inner exception stack trace ---
   at MQTTnet.Adapter.MqttChannelAdapter.WrapException(Exception exception) in C:\Users\lbostock\Source\Repos\MQTTnet\Source\MQTTnet\Adapter\MqttChannelAdapter.cs:line 315
   at MQTTnet.Adapter.MqttChannelAdapter.ConnectAsync(TimeSpan timeout, CancellationToken cancellationToken) in C:\Users\lbostock\Source\Repos\MQTTnet\Source\MQTTnet\Adapter\MqttChannelAdapter.cs:line 80
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken) in C:\Users\lbostock\Source\Repos\MQTTnet\Source\MQTTnet\Client\MqttClient.cs:line 81
[17:43:41.818 VRB] MqttNetLogEvent. LogId: null, Message: Disconnecting [Timeout=00:00:10], Source: MqttClient, ThreadId: 5, Timestamp: 05/28/2019 16:43:41

I pulled in the source code from master for mqttnet and can see that it's hanging at this call:

        private async Task DisconnectInternalAsync(Task sender, Exception exception, MqttClientAuthenticateResult authenticateResult)
        {
            var clientWasConnected = IsConnected;

            InitiateDisconnect();

            IsConnected = false;

            try
            {
                if (_adapter != null)
                {
                    _logger.Verbose("Disconnecting [Timeout={0}]", Options.CommunicationTimeout);
                    await _adapter.DisconnectAsync(Options.CommunicationTimeout, CancellationToken.None).ConfigureAwait(false);
                }

                /*HANGS HERE >>>>>*/ await WaitForTaskAsync(_packetReceiverTask, sender).ConfigureAwait(false);
                await WaitForTaskAsync(_keepAlivePacketsSenderTask, sender).ConfigureAwait(false);

                _logger.Verbose("Disconnected from adapter.");
            }
            catch (Exception adapterException)
            {
                _logger.Warning(adapterException, "Error while disconnecting from adapter.");
            }
            finally
            {
                Dispose();
                _cleanDisconnectInitiated = false;

                _logger.Info("Disconnected.");

                var disconnectedHandler = DisconnectedHandler;
                if (disconnectedHandler != null)
                {
                    await disconnectedHandler.HandleDisconnectedAsync(new MqttClientDisconnectedEventArgs(clientWasConnected, exception, authenticateResult)).ConfigureAwait(false);
                }
            }
        }

I'll be the first to admit that I'm not completely comfortable with async/await, hence me trying .ConfigureAwait(false) on the calls, but I'm really at a loss.

I suppose it's also worth noting, we have a spike which was written against MQTTNet 2.8.5, the code is near identical apart from it's an event handler:

            _client.Disconnected += async (s, e) =>
            {
                OpenCircuitBreaker();

                // Disconnected event is fired when DisconnectAsync is called.  Don't want to reconnect when the service is stopping
                if (_isDisconnecting)
                {
                    return;
                }

                _logger.LogWarning(EventIds.ConnectionLost, "Connection lost. Attempting to reconnect in {ReconnectBackoffSeconds} milliseconds",
                    _config.ReconnectBackoffMilliseconds);

                await Task.Delay(TimeSpan.FromMilliseconds(_config.ReconnectBackoffMilliseconds));
                await ConnectInternalAsync();
            };

This works fine.

bug

Source

lizziebeans

👍1

Most helpful comment

@milamber-ls a better formatting would make your posts more readable, please take a look at here: https://help.github.com/en/articles/creating-and-highlighting-code-blocks

you are right! just edited the post

milamber-ls on 1 Jul 2019

👍2

All 21 comments

Another thing I've noticed is that all of the ####### Reconnect failed ####### are logged all at once _after_ the application has reconnected.
This obviously only works if I start the application with the message broker off.

lizziebeans on 29 May 2019

👍1

Could you do me a favor and change line

_await disconnectedHandler.HandleDisconnectedAsync(new MqttClientDisconnectedEventArgs(clientWasConnected, exception, authenticateResult)).ConfigureAwait(false);_

to:

_Task.Run(disconnectedHandler.Handle...)_ without(!) awaiting the task.

I am afraid the reconnecting in the disconnected handler will not work anymore because since 3.0.0 the calls are synchronized.

chkr1011 on 31 May 2019

@chkr1011 thanks for the reply.
Yeah, doing that makes it work, but it also generates a compiler warning:

CS4014 C# Because this call is not awaited, execution of the current method continues before the call is completed. Consider applying the 'await' operator to the result of the call.

Where do we go from here then? Is there a better way of reconnecting?

It's also worth noting that reconnecting this way is detailed in the documentation:
https://github.com/chkr1011/MQTTnet/wiki/Client#reconnecting

lizziebeans on 3 Jun 2019

OK that is perfect. I will implement it properly so that the warning will be also gone.

The example in the Wiki is correct. It shows the supposed way of handling this.

chkr1011 on 3 Jun 2019

👍1

Awesome.
Just an FYI really, but we decided to go with

            _mqttClient.UseDisconnectedHandler(e =>
            {
                Task.Run(() => Retry(_mqttSessionConfig.ReconnectBackoffMilliseconds));
            });

        private async Task Retry(int retryInMilliseconds)
        {
            _logger.LogWarning(EventIds.Reconnecting, $"Retrying in {retryInMilliseconds} ms");
            await Task.Delay(TimeSpan.FromMilliseconds(retryInMilliseconds));

            try
            {
                await _mqttClient.ConnectAsync(_clientOptions);
            }
            catch(Exception ex)
            {
                _logger.LogError(EventIds.ReconnectFailed, ex, "Reconnection failed");
            }
        }

as a work around, for now.

lizziebeans on 6 Jun 2019

I released a new nuget _3.0.3-rc2_. Please let me know if this version works. You don't need to use the Task.Run anymore.

chkr1011 on 6 Jun 2019

I got the same issue with 3.0.2 and I can confirm that is solved with 3.0.3-rc2.
Let's wait for @lizziebeans confirmation anyway

paolofulgoni on 14 Jun 2019

Sorry for the super late reply boys.
Just upgraded to 3.0.3-rc3 and can confirm that it's working.

lizziebeans on 18 Jun 2019

I am not sure the the problem is solved at version >3.0 the wiki example of how to reconnect

client.UseDisconnectedHandler(async e =>
{
    Console.WriteLine("### DISCONNECTED FROM SERVER ###");
    await Task.Delay(TimeSpan.FromSeconds(5));

    try
    {
        await client.ConnectAsync(options);
    }
    catch
    {
        Console.WriteLine("### RECONNECTING FAILED ###");
    }
});

doesnt work. I have tried all minor version > 3.0.3-rc3. most of the times it doesnt reconnect. some times it might reconnect minutes after the server is online but the "server is connected' event doesnt fire

here is my code

```async Task OnDisconnected(MqttClientDisconnectedEventArgs e)
{
Console.WriteLine("### DISCONNECTED FROM SERVER ###");
Console.WriteLine("### TRY AFTER 5 SECONDS ###");
while (!client.IsConnected)
{
await Task.Delay(TimeSpan.FromSeconds(10));
Console.Write(".");
try
{
await client.ConnectAsync(options);
await client.SubscribeAsync(new TopicFilterBuilder().WithTopic("sensors/+/data").WithAtLeastOnceQoS().Build());
Console.WriteLine("### RECONNECTING SUCCESS ###");
}
catch
{
Console.WriteLine("### RECONNECTING FAILED ###");
}
}
}


and this is the output when server is off and then on. Also doesnt wait for 5 seconds but rather fires it is rapid succession

RECONNECTING FAILED ###

RECONNECTING FAILED

............### RECONNECTING FAILED ###

RECONNECTING FAILED

....................### RECONNECTING FAILED ###

RECONNECTING FAILED

.............### RECONNECTING FAILED ###

RECONNECTING FAILED

..........................### RECONNECTING FAILED ###

RECONNECTING FAILED

...........### RECONNECTING FAILED ###

RECONNECTING FAILED

.......### RECONNECTING FAILED ###

RECONNECTING FAILED

...### RECONNECTING FAILED ###

RECONNECTING FAILED

................................................### RECONNECTING FAILED ###

RECONNECTING FAILED

.### RECONNECTING FAILED ###

RECONNECTING FAILED

.......................### RECONNECTING FAILED ###
.......### RECONNECTING FAILED ###

```

milamber-ls on 29 Jun 2019

Please add the exception detail from the catch clause. It might contain relevant data.

chkr1011 on 29 Jun 2019

I got multiple exceptions

MQTTnet.Exceptions.MqttCommunicationException: No connection could be made because the target machine actively refused it [::ffff:222.222.222.222]:8883 ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: No connection could be made because the target machine actively refused it [::ffff:222.222.222.222]:8883
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.Socket.DoMultipleAddressConnectCallback(Object result, MultipleAddressConnectAsyncResult context)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Sockets.Socket.DoMultipleAddressConnectCallback(Object result, MultipleAddressConnectAsyncResult context)
   at System.Net.Sockets.Socket.MultipleAddressConnectCallback(IAsyncResult result)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.Socket.<>c.<ConnectAsync>b__274_0(IAsyncResult iar)
--- End of stack trace from previous location where exception was thrown ---
   at MQTTnet.Implementations.MqttTcpChannel.ConnectAsync(CancellationToken cancellationToken)
   at MQTTnet.Internal.MqttTaskTimeout.WaitAsync(Func`2 action, TimeSpan timeout, CancellationToken cancellationToken)
   at MQTTnet.Adapter.MqttChannelAdapter.ConnectAsync(TimeSpan timeout, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at MQTTnet.Adapter.MqttChannelAdapter.WrapException(Exception exception)
   at MQTTnet.Adapter.MqttChannelAdapter.ConnectAsync(TimeSpan timeout, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)


   MQTTnet.Exceptions.MqttCommunicationException:  The ReadAsync method cannot be called when another read operation is pending. ---> System.NotSupportedException:  The ReadAsync method cannot be called when another read operation is pending.
   at System.Net.Security.SslStreamInternal.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory`1 buffer)
   at MQTTnet.Implementations.MqttTcpChannel.ReadAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at MQTTnet.Formatter.MqttPacketReader.ReadFixedHeaderAsync(Byte[] fixedHeaderBuffer, CancellationToken cancellationToken)
   at MQTTnet.Adapter.MqttChannelAdapter.ReceiveAsync(CancellationToken cancellationToken)
   at MQTTnet.Adapter.MqttChannelAdapter.ReceivePacketAsync(TimeSpan timeout, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at MQTTnet.Adapter.MqttChannelAdapter.WrapException(Exception exception)
   at MQTTnet.Adapter.MqttChannelAdapter.ReceivePacketAsync(TimeSpan timeout, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.TryReceivePacketsAsync(CancellationToken cancellationToken)
   at MQTTnet.PacketDispatcher.MqttPacketAwaiter`1.WaitOneAsync(TimeSpan timeout)
   at MQTTnet.Client.MqttClient.SendAndReceiveAsync[TResponsePacket](MqttBasePacket requestPacket, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.AuthenticateAsync(IMqttChannelAdapter channelAdapter, MqttApplicationMessage willApplicationMessage, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)

   System.InvalidOperationException: The packet dispatcher already has an awaiter for packet of type 'MqttConnAckPacket' with identifier 0.
   at MQTTnet.PacketDispatcher.MqttPacketDispatcher.AddPacketAwaiter[TResponsePacket](Nullable`1 identifier)
   at MQTTnet.Client.MqttClient.SendAndReceiveAsync[TResponsePacket](MqttBasePacket requestPacket, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.AuthenticateAsync(IMqttChannelAdapter channelAdapter, MqttApplicationMessage willApplicationMessage, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)

   MQTTnet.Exceptions.MqttCommunicationException: Object reference not set to an instance of an object. ---> System.NullReferenceException: Object reference not set to an instance of an object.
   at MQTTnet.Implementations.MqttTcpChannel.WriteAsync(Byte[] buffer, Int32 offset, Int32 count, CancellationToken cancellationToken)
   at MQTTnet.Internal.MqttTaskTimeout.WaitAsync(Func`2 action, TimeSpan timeout, CancellationToken cancellationToken)
   at MQTTnet.Adapter.MqttChannelAdapter.SendPacketAsync(MqttBasePacket packet, TimeSpan timeout, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at MQTTnet.Adapter.MqttChannelAdapter.WrapException(Exception exception)
   at MQTTnet.Adapter.MqttChannelAdapter.SendPacketAsync(MqttBasePacket packet, TimeSpan timeout, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.SendAndReceiveAsync[TResponsePacket](MqttBasePacket requestPacket, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.AuthenticateAsync(IMqttChannelAdapter channelAdapter, MqttApplicationMessage willApplicationMessage, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)

   System.NullReferenceException: Object reference not set to an instance of an object.
   at MQTTnet.Client.MqttClient.SendAndReceiveAsync[TResponsePacket](MqttBasePacket requestPacket, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.AuthenticateAsync(IMqttChannelAdapter channelAdapter, MqttApplicationMessage willApplicationMessage, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)

   MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)

milamber-ls on 29 Jun 2019

the end result of the above is message loss when qos >0

milamber-ls on 29 Jun 2019

@milamber-ls a better formatting would make your posts more readable, please take a look at here: https://help.github.com/en/articles/creating-and-highlighting-code-blocks

paolofulgoni on 30 Jun 2019

😄1 👍1

@milamber-ls a better formatting would make your posts more readable, please take a look at here: https://help.github.com/en/articles/creating-and-highlighting-code-blocks

you are right! just edited the post

milamber-ls on 1 Jul 2019

👍2

any update on this?

milamber-ls on 11 Jul 2019

@milamber-ls Not yet, someone needs to check this. I don't know when I will be able to do so though^^

SeppPenner on 11 Jul 2019

@milamber-ls I added some UnitTests like Reconnect_From_Disconnected_Event. Please try to modify them so that they fail as well. At the moment I cannot reproduce the issue.

Also please try again with the latest beta nuget. Maybe the issue is already fixed.

chkr1011 on 30 Jul 2019

I have tried with the lasted beta. the issue has been improved but still throws errors. I will modify your unit tests and sent them

mqtt server stopped <=====
### RECONNECTING FAILED ###
System.AggregateException: One or more errors occurred. (No connection could be made because the target machine actively refused it [::ffff:159.203.184.81]:8883) ---> MQTTnet.Exceptions.MqttCommunicationException: No connection could be made because the target machine actively refused it [::ffff:159.203.184.81]:8883 ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: No connection could be made because the target machine actively refused it [::ffff:159.203.184.81]:8883
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.Socket.DoMultipleAddressConnectCallback(Object result, MultipleAddressConnectAsyncResult context)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Sockets.Socket.DoMultipleAddressConnectCallback(Object result, MultipleAddressConnectAsyncResult context)
   at System.Net.Sockets.Socket.MultipleAddressConnectCallback(IAsyncResult result)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.Socket.<>c.<ConnectAsync>b__274_0(IAsyncResult iar)
--- End of stack trace from previous location where exception was thrown ---
   at MQTTnet.Implementations.MqttTcpChannel.ConnectAsync(CancellationToken cancellationToken)
   at MQTTnet.Internal.MqttTaskTimeout.WaitAsync(Func`2 action, TimeSpan timeout, CancellationToken cancellationToken)
   at MQTTnet.Adapter.MqttChannelAdapter.ConnectAsync(TimeSpan timeout, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at MQTTnet.Adapter.MqttChannelAdapter.WrapException(Exception exception)
   at MQTTnet.Adapter.MqttChannelAdapter.ConnectAsync(TimeSpan timeout, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at XXXXXXXX.mqtt.MqttSubscriber.OnDisconnected(MqttClientDisconnectedEventArgs e) in C:\d\LS\XXXXXXXX.mqtt\mqttSubscriber\MqttSubscriber.cs:line 318
---> (Inner Exception #0) MQTTnet.Exceptions.MqttCommunicationException: No connection could be made because the target machine actively refused it [::ffff:159.203.184.81]:8883 ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: No connection could be made because the target machine actively refused it [::ffff:159.203.184.81]:8883
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.Socket.DoMultipleAddressConnectCallback(Object result, MultipleAddressConnectAsyncResult context)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Sockets.Socket.DoMultipleAddressConnectCallback(Object result, MultipleAddressConnectAsyncResult context)
   at System.Net.Sockets.Socket.MultipleAddressConnectCallback(IAsyncResult result)
--- End of stack trace from previous location where exception was thrown ---
   at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
   at System.Net.Sockets.Socket.<>c.<ConnectAsync>b__274_0(IAsyncResult iar)
--- End of stack trace from previous location where exception was thrown ---
   at MQTTnet.Implementations.MqttTcpChannel.ConnectAsync(CancellationToken cancellationToken)
   at MQTTnet.Internal.MqttTaskTimeout.WaitAsync(Func`2 action, TimeSpan timeout, CancellationToken cancellationToken)
   at MQTTnet.Adapter.MqttChannelAdapter.ConnectAsync(TimeSpan timeout, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at MQTTnet.Adapter.MqttChannelAdapter.WrapException(Exception exception)
   at MQTTnet.Adapter.MqttChannelAdapter.ConnectAsync(TimeSpan timeout, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)<---

..### RECONNECTING FAILED ###
System.AggregateException: One or more errors occurred. (A task was canceled.) ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at XXXXXXXX.mqtt.MqttSubscriber.OnDisconnected(MqttClientDisconnectedEventArgs e) in C:\d\LS\XXXXXXXX.mqtt\mqttSubscriber\MqttSubscriber.cs:line 318
---> (Inner Exception #0) System.Threading.Tasks.TaskCanceledException: A task was canceled.<---

### DISCONNECTED FROM SERVER ###
### TRY AFTER 5 SECONDS ###
### RECONNECTING FAILED ###
System.AggregateException: One or more errors occurred. (The packet dispatcher already has an awaiter for packet of type 'MqttConnAckPacket' with identifier 0.) ---> System.InvalidOperationException: The packet dispatcher already has an awaiter for packet of type 'MqttConnAckPacket' with identifier 0.
   at MQTTnet.PacketDispatcher.MqttPacketDispatcher.AddPacketAwaiter[TResponsePacket](Nullable`1 identifier)
   at MQTTnet.Client.MqttClient.SendAndReceiveAsync[TResponsePacket](MqttBasePacket requestPacket, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.AuthenticateAsync(IMqttChannelAdapter channelAdapter, MqttApplicationMessage willApplicationMessage, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at XXXXXXXX.mqtt.MqttSubscriber.OnDisconnected(MqttClientDisconnectedEventArgs e) in C:\d\LS\XXXXXXXX.mqtt\mqttSubscriber\MqttSubscriber.cs:line 318
---> (Inner Exception #0) System.InvalidOperationException: The packet dispatcher already has an awaiter for packet of type 'MqttConnAckPacket' with identifier 0.
   at MQTTnet.PacketDispatcher.MqttPacketDispatcher.AddPacketAwaiter[TResponsePacket](Nullable`1 identifier)
   at MQTTnet.Client.MqttClient.SendAndReceiveAsync[TResponsePacket](MqttBasePacket requestPacket, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.AuthenticateAsync(IMqttChannelAdapter channelAdapter, MqttApplicationMessage willApplicationMessage, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)<---

.### CONNECTED WITH SERVER ###
..### RECONNECTING FAILED ###
System.AggregateException: One or more errors occurred. (It is not allowed to connect with a server after the connection is established.) ---> MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at XXXXXXXX.mqtt.MqttSubscriber.OnDisconnected(MqttClientDisconnectedEventArgs e) in C:\d\LS\XXXXXXXX.mqtt\mqttSubscriber\MqttSubscriber.cs:line 318
---> (Inner Exception #0) MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)<---

### RECONNECTING FAILED ###
System.AggregateException: One or more errors occurred. (It is not allowed to connect with a server after the connection is established.) ---> MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at XXXXXXXX.mqtt.MqttSubscriber.OnDisconnected(MqttClientDisconnectedEventArgs e) in C:\d\LS\XXXXXXXX.mqtt\mqttSubscriber\MqttSubscriber.cs:line 318
---> (Inner Exception #0) MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)<---
mqtt server started <========
### SUBSCRIBED ###
### RECONNECTING SUCCESS ###
.### RECONNECTING FAILED ###
System.AggregateException: One or more errors occurred. (It is not allowed to connect with a server after the connection is established.) ---> MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at XXXXXXXX.mqtt.MqttSubscriber.OnDisconnected(MqttClientDisconnectedEventArgs e) in C:\d\LS\XXXXXXXX.mqtt\mqttSubscriber\MqttSubscriber.cs:line 318
---> (Inner Exception #0) MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)<---

...### RECONNECTING FAILED ###
System.AggregateException: One or more errors occurred. (It is not allowed to connect with a server after the connection is established.) ---> MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at XXXXXXXX.mqtt.MqttSubscriber.OnDisconnected(MqttClientDisconnectedEventArgs e) in C:\d\LS\XXXXXXXX.mqtt\mqttSubscriber\MqttSubscriber.cs:line 318
---> (Inner Exception #0) MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)<---

### RECONNECTING FAILED ###
System.AggregateException: One or more errors occurred. (It is not allowed to connect with a server after the connection is established.) ---> MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at XXXXXXXX.mqtt.MqttSubscriber.OnDisconnected(MqttClientDisconnectedEventArgs e) in C:\d\LS\XXXXXXXX.mqtt\mqttSubscriber\MqttSubscriber.cs:line 318
---> (Inner Exception #0) MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)<---

### RECONNECTING FAILED ###
System.AggregateException: One or more errors occurred. (It is not allowed to connect with a server after the connection is established.) ---> MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
   at XXXXXXXX.mqtt.MqttSubscriber.OnDisconnected(MqttClientDisconnectedEventArgs e) in C:\d\LS\XXXXXXXX.mqtt\mqttSubscriber\MqttSubscriber.cs:line 318
---> (Inner Exception #0) MQTTnet.Exceptions.MqttProtocolViolationException: It is not allowed to connect with a server after the connection is established.
   at MQTTnet.Client.MqttClient.ConnectAsync(IMqttClientOptions options, CancellationToken cancellationToken)<---

milamber-ls on 8 Aug 2019

ignore my previous post. i had changed the reconnect code a bit trying to find a work around. changing the code back to the example and using 3.0.6 beta 1 solved the issue. the issue from my end can be closed

milamber-ls on 8 Aug 2019

@milamber-ls Thanks for the information. I will close this now. If anyone still has problems with this issue, hit me up here and I will re-open it.

SeppPenner on 8 Aug 2019

I am having the same problem. Did you find a solution?