Stackexchange.redis: ArgumentException: Item has already been added. Key in dictionary: '127.0.0.1:6379'

[odinserj]

After upgrading to the newest versions (both 1.1.605 and 1.1.604-alpha are affected), I've started to receive the following exception from time to time, when using .NET Framework 4.5 on my development machine (I've got it on Appveyor also).

System.AggregateException: One or more errors occurred. ---> System.ArgumentException: Item has already been added. Key in dictionary: '127.0.0.1:6379'  Key being added: '127.0.0.1:6379'
   at System.Collections.Hashtable.Insert(Object key, Object nvalue, Boolean add)
   at System.Collections.Hashtable.Add(Object key, Object value)
   at StackExchange.Redis.ConnectionMultiplexer.<ReconfigureAsync>d__120.MoveNext()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at StackExchange.Redis.ConnectionMultiplexer.ConnectImpl(Func`1 multiplexerFactory, TextWriter log)
   at StackExchange.Redis.ConnectionMultiplexer.Connect(String configuration, TextWriter log)

The following code snippet may be used to reproduce the error, we can even use a single thread to get it:

for (var i = 0; i < 1000; i++)
{
    ConnectionMultiplexer.Connect("127.0.0.1");
}

There is the #148 issue reported in 2015 with the same exception.

[ProVega]

This just started happening to use. We are using StackExchange.Redis.StrongName 1.1.605.0, previously never saw this in 1.1.603

System.AggregateException: One or more errors occurred. ---> System.ArgumentException: Item has already been added. Key in dictionary: '10.0.5.41:5533' Key being added: '10.0.5.41:5533'

[wianek]

It is also happening to me on my local machine app most of the times when I start it. I'm using lazy initialization of the ConnectionMultiplexer.

[c-mccutcheon]

Also started happening to me after upgrade to the most recent stable package. Never had this issue before but upgraded to fix a previous client connection leak problem.

Can confirm am not seeing the error when the solution is downgraded to 1.1.603

[mgravell]

I have uploaded 606 which should address this issue.

On 4 October 2016 at 14:26, omgsarge [email protected] wrote:

Also started happening to me after upgrade to the most recent stable
package. Never had this issue before but upgraded to fix a previous client
connection leak problem.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/StackExchange/StackExchange.Redis/issues/495#issuecomment-251386857,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABDsEIjiD1nKn314CdESfvLwXVNv1Coks5qwlP1gaJpZM4KHd10
.

Regards,

Marc

[mgravell]

link: https://www.nuget.org/packages/StackExchange.Redis/1.1.606

We have successfully dogfooded it internally; I'm releasing it to public now.

[odinserj]

@mgravell, the issue still exists in 1.1.606, I was able to reproduce it with the same code snippet. I also was able to resolve the issue by using the double checked locking, as in the GetServerEndPoint method. I'll submit a PR with the fix in few minutes.

UPD. That section is already locked with lock (servers), so double checked locking adds nothing there.

[odinserj]

@mgravell, this problem exists since the commit dce0f8ffbb3b68e72b2502efe9f07e9a639afd84. In that commit the FEATURE_SOCKET_MODE_POLL define was added to have separate implementations for .NET Core and .NET Framework. I don't know yet how do you build assemblies that are shipped with NuGet packages, but looks like that the FEATURE_SOCKET_MODE_POLL define isn't specified during the build process. For example, this define is absent in netbuild.cmd.

When this define is absent, builds for .NET Framework 4.x begin to use SocketMode.Async, instead of SocketMode.Poll as in versions 1.1.603 and below. I've decompiled assemblies that are shipped with NuGet packages, and found the confirmation:

lib/net45/StackExchange.Redis.dll 1.1.603

lib/net45/StackExchange.Redis.dll 1.1.606 (1.1.605, 1.1.604-alpha has the same code)

Since the problem appears when we are using SocketMode.Async, I've also tested the library against .NET Core on Windows, but was unable to reproduce the exception. So we only need to fix assemblies that target full framework to use SocketModel.Poll.

I see two ways to resolve the issue:

1. Invert the FEATURE_SOCKET_MODE_POLL define to not to require it for building correct packages for full framework.
2. Include that define in build scripts explicitly – all the *.csproj files already contain it.

[mgravell]

Re the SocketMode.Poll - (face/desk) - that's a major fail by me; it
shouldn't cause this bug, but that needs fixing indeed.

Re the double-checked locking; the relevant code _is_ already in a lock;
note that I'm currently working in the core-rtm branch (note to self; I
need to merge that down). Is it possible you used source from "master" for
your compare?

[odinserj]

Re the double-checked locking; the relevant code _is_ already in a lock; note that I'm currently working in the core-rtm branch (note to self; I need to merge that down). Is it possible you used source from "master" for your compare?

Yep, I've already seen there is a lock, please ignore that comment. I've also realized that this exception is caused by ServerEndPoint constructor that may modify the servers collection, so the following lines need an additional check:

server = new ServerEndPoint(this, endpoint, null); 
if (servers[endpoint] == null) 
{ 
    servers.Add(endpoint, server); 
}

ServerEndPoint constructor causes the connection to be established with an extra ReconfigureAsync call (with the Databases reason), causing GetServerEndpoint (that modifies the collection) to be called.

But I stopped the investigation, because found out that the problem is caused by the code leaked from the implementation for .NET Core. And I don't know yet whether this check is enough, or there may be other race conditions as well.

[mgravell]

Gaddamit re-entrancy! Fixing. Good eyes, internet person.

[mgravell]

Build 1.1.608 is available on nuget

[odinserj]

Checked against .NET Framework 4.5 on Windows and .NET Core on Windows, Linux and macOS :tada:. Looks like everything is fine now, thanks for your help!