StackExchange.Redis async call hangs

[ejsmith]

I am seeing a weird issue that is causing my code to hang when I do a certain sequence of events. I am not sure what is going on. Any help would be appreciated.

http://stackoverflow.com/questions/27258984/strange-stackexchange-redis-async-issue

[nmehlei]

Yes this is a known problem, in some (async) situations the TPL "hijacks" an internal StackExchange.Redis thread and this results in slowness and hangs, as seen in https://github.com/StackExchange/StackExchange.Redis/issues/88 and via google "StackExchange.Redis TPL hijack"

I'm not sure but I think I resolved it in my code by switching to Sync methods in a few cases and ConfigureAwait(false) in others.

[ejsmith]

Ugh. I just spent a ton of time changing all my code to be async (which is a quite viral process). Now I'm not sure what to do. From everything I'm reading and trying, there doesn't seem to be a work around. Any ideas on how this can be fixed?

[nmehlei]

I feel you, I went through more or less the same thing. Unfortunately, I am not exactly sure what I changed to "fix" this in my codebase, was a lot of fiddling around until it never happened.

To rule something out, are you sure you aren't deadlock'ing yourself?
http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html
VERY easy with await, maybe you just need a few ConfigureAwait(false) in your code :)

[ejsmith]

I can try restructuring my code to see if that helps, but I have created a pretty simple repro that shows the issue. I am going to submit a unit test pull request. Hopefully that will help.

[mwikstrom]

I think that there is a problem causing deadlocks _inside_ library code.

Please see this question on StackOverflow:
http://stackoverflow.com/questions/30797716/deadlock-when-accessing-stackexchange-redis

[operezfuentes]

+1 to @mwikstrom
We are reporting this issue during last 4 months and it's crazy to reproduce.To avoid it we are recycling app pools every 1-2 hours....
One alternative should be move to ServiceStack.Redis but a license is required and still we love StackExchange.Redis

[p-kaczynski]

We have encountered this issue recently - after few moments under medium/heavy load, all async calls to Redis seem to hang indefinitely. Web API 2 on IIS8.5@WindowsServer2012R2.

However, this only started after changing build agent from MSBuild 2013 to 2015 - and seems to have disappeared after reverting back to 2013.

[Funbit]

I can confirm the issue. I can also confirm that it does not relate to the MSBuild version, both 2013 and 2015 can cause the same deadlock. We have experienced the problem for third time now (within 6 months), so we can leave with it for now, but...
BTW. We are using sync version of the library methods. The stack trace of the dumped process hangs here: http://postimg.org/image/64f6tjoej/

[keith-tingle]

I can confirm this issue as well. Web API 2, Azure Redis, Azure Cloud Service Web Role W12R2. We would encounter this issue intermittently with just 2-3 QA users on our system. HTTP requests to IIS would receive no response and the request would just hang indefinitely.

We were mixing various async calls with a single synchronous HashGetAll(). We also use the SignalR Redis backplane in our configuration.

As a workaround we lowered the PreserveAsyncOrder flag and migrated to 100% asynchronous methods.

[EnCey]

I think I might have run into this problem; did anyone find a cause / solution?

[keith-tingle]

In my case, I was blending async / await style calls into the library with synchronous calls. I removed the synchronous calls from my code and the deadlocks stopped occurring.

[EnCey]

This StackOverflow question ended up helping me.

Setting PreserveAsyncOrder to false prevents deadlocks in my case, but I'm also going to remove all sync-over-async calls that we still have in some cases (still an issue despite there not being a SynchronizationContext).
There's also an "official" recommendation to turn this off on the StackExchange.Redis page.

[AndrewGumenyuk]

@EnCey thanks a lot. For me, it was also huge set of deadlocks

[NickCraver]

Question answered in here, so closing this out to cleanup :) For those finding this later, the Stack Overflow answer by Marc's a good place to start.