Aspnetcore.docs: SignalR Scale-out Doc

[anurse]

We need a doc that describes why Scale-Out providers are needed in SignalR, the two main options (Redis and Azure SignalR Service), and how to configure them.

[anurse]

See:

• https://github.com/aspnet/SignalR/issues/2671 (questions on Redis config)
• https://github.com/aspnet/Docs/issues/7204
• https://github.com/aspnet/Docs/issues/7450

[Rick-Anderson]

@anurse can you draft the doc?

[anurse]

@Rick-Anderson someday, sure :)

[anurse]

As part of this, we'll want to show how to hook the disconnect event in case a user wants to add custom behavior when the connection disconnects:

```c#
services.AddSignalR()
.AddMessagePackProtocol()
.AddRedis(o =>
{
o.ConnectionFactory = async writer =>
{
var config = new ConfigurationOptions
{
AbortOnConnectFail = false
};
config.EndPoints.Add(IPAddress.Loopback, 0);
config.SetDefaultPorts();
var connection = await ConnectionMultiplexer.ConnectAsync(config, writer);
connection.ConnectionFailed += (_, e) =>
{
Console.WriteLine("Connection Redis failed.");
};

            if (!connection.IsConnected)
            {
                Console.WriteLine("Connection did not connect.");
            }

            return connection;
        };
    });

```

(source https://github.com/aspnet/SignalR/issues/2468#issuecomment-414431147)

[scottaddie]

@anurse Scale-out with SQL Server won't be supported in ASP.NET Core. Since this was supported in ASP.NET 4.x, should we add a "Scale-out differences" section to the Version differences doc?

[anurse]

Sure, that sounds good

[davidfowl]

@anurse we should add a pointer to this doc from here https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/web-farm?view=aspnetcore-2.1

[anurse]

Some technical notes and a couple images stolen from a PowerPoint I dug up to help illustrate.

• SignalR applications scale differently from traditional web apps
• Each SignalR connection takes up resources on the system even when idle
  
  
  
  • Memory required to track the connection
  
  
  • TCP connections that live for the entire duration of the connection (unlike in HTTP where connections can be reused for multiple requests)
  
  
  • A large number of idle persistent connections could block out other HTTP traffic
  
  
• However, SignalR applications usually have fairly moderate traffic (depending on the app) since often they are just pushing small notifications to clients.
• For high-traffic applications, we recommend running SignalR apps separate from traditional web apps (i.e. put your real-time SignalR code on a separate set of servers)
  
  
  
  • This prevents SignalR traffic from "starving" other HTTP traffic of connection resources
  
  
• The Azure SignalR Service makes this easy by off-loading the persistent connections to the managed service while still having your app handle the individual messages
  
  
  
  • You can scale your app based on the actual throughput rather than the number of concurrnt connections
  
  
• If you choose to scale your app manually you need a backplane:
  
  
  
  • SignalR Connections are sticky to a single server. If you have multiple servers, a server can only send messages to it's own clients:
    
    
    
    
    
    • 
    
    
    
  
  
  • When you have multiple servers, you need a "backplane" to ensure that when one server wants to send a message to all clients, all the other servers get the message so they can forward it to their clients:
    
    
    
    
    
    • 
    
    
    
  
  
  • SignalR provides one backplane implementation based on Redis
  
  
  • To use it, add NuGet reference to Microsoft.AspNetCore.SignalR.StackExchangeRedis (new in 2.2; we have a different package for 2.1 but we want to guide users to the new one) on the server
  
  
  • Deploy a Redis server (or use Azure Redis Cache)
  
  
  • Call .AddRedis() after .AddSignalR() in ConfigureServices:
  
  

public void ConfigureServices(IServiceCollection services)
{
    // ... other services ...
    services.AddSignalR().AddStackExchangeRedis(redisConnectionString);
    // ... other services ...
}

Configure Redis options by passing a second argument (a delegate that can modify the RedisOptions):

public void ConfigureServices(IServiceCollection services)
{
    // ... other services ...
    services.AddSignalR().AddStackExchangeRedis(redisConnectionString, options => 
    {
        // change Redis options here
    });
    // ... other services ...
}

• Redis Options:
  
  
  
  • Configuration - Provides access to StackExchange.Redis's ConfigurationOptions object. Most of these options can also be provided in the connection string. Options configured here override the ones set in the connection string.
  
  
  • ConnectionFactory - Allows you to provide you own custom logic to create a ConnectionMultiplexer to use as the connection information. Receives a TextWriter which can be passed in to the ConnectionMultiplexer so that log messages will be forwarded.
  
  
• Common Redis configuration scenarios:
  
  
  
  • Change the ChannelPrefix so that multiple applications can share one Redis server.
  
  
  • Provide the password separate from the connection string
  
  

[csatnic-av]

In https://github.com/aspnet/SignalR/issues/3193 @anurse said this:

We don't have SignalR-specific guidance around Redis high-availability. I'd suggest looking at some Redis-specific guidance here. SignalR only uses pub/sub so you don't really need any 'durability' guarantees around your data, you just need to make sure all the servers get all the messages.

We don't officially support clustering (as you noted, there is very limited documentation on Redis right now, this is being worked on in aspnet/Docs#7814). I don't know for sure that it doesn't work, but we don't generally test in that scenario.

So ... Redis clustering is not officially supported. All existing Core SignalR documentation seems to only reference a single Redis server on the backplane.

I understand that only Redis pub/sub functionality is being used so what happens when that Redis server needs to be maintained, patched, restarted? Are SignalR messages only going to be sent to clients attached to the server node that sent the message? (instead of all server nodes). Will SignalR properly reconnect to the Redis server when it comes back after the restart?

If high-availability for Redis is not something that's officially supported or tested I'm trying to determine the implications of having that Redis server as a single-point-of-failure resource. This is something that should be documented & explained since I see many others are trying to find the answer to this exact question.

[neildlg]

I've been wondering about the same things that @csatnic-av asks in his comment above - how are we meant to deploy Redis in a way that it can be used as a backplane and also ensure high availability?

As @csatnic-av says:

So ... Redis clustering is not officially supported. All existing Core SignalR documentation seems to only reference a single Redis server on the backplane.

I understand that only Redis pub/sub functionality is being used so what happens when that Redis server needs to be maintained, patched, restarted? Are SignalR messages only going to be sent to clients attached to the server node that sent the message? (instead of all server nodes). Will SignalR properly reconnect to the Redis server when it comes back after the restart?

If high-availability for Redis is not something that's officially supported or tested I'm trying to determine the implications of having that Redis server as a single-point-of-failure resource. This is something that should be documented & explained since I see many others are trying to find the answer to this exact question.

I've read the new documentation here and here but it doesn't seem to address this question, even though it was written in response to this issue.

Since I'm working in Kubernetes I've also tried the Sentinel master-slave topology using the Binnami Redis Helm Chart, but this does not seem usable for the backplane as it deploys separate Kubernetes services for master (writing) and slave (reading), whereas the C# extension method AddStackExchangeRedis(), which sets up the backplane, only allows a single Redis connection string for both reading and writing. So it does not seem possible to use this topology either.