Shiny: Feature request: Have client send keep alive message to prevent TCP timeout on some load balancers

[shimberger]

We have deployed a shiny app behind an AWS Elastic Load Balancer. We have issues that the app disconnects after some time of inactivity because the load balancer disconnects the idle TCP connection. It would be nice to have a configuration option in shiny::runApp to have the client send a ping or keep-alive message every n seconds. This could just be a JSON message that get's send over the WebSocket in the form of

{
  method: 'ping'
  data: null,
}

or something like this. It doesn't sound hard to implement (I am familiar with JavaScript but not R) and would be willing to spend a few hours doing so if there is interest in that feature and I would be given some hints on where to start? E.g. add it in srcjs/shinyapp.js and somewhere in the R code.

Thanks.

If this is not interesting, should be solved otherwise etc. Please just tell me and close the ticket. I know there is always too much stuff to do ;)

[alandipert]

Can you please elaborate a little more on what you see during and after the disconnect?

I ask because if you're on an ELB, you must be using Shiny Server or Connect, since only those offer the ability to use non-WebSocket transports. And ELBs don't supports WebSockets.

So, I suspect what you're seeing is probably not related to WebSockets, but might be solved by other means, such as further configuration of Shiny Server/Connect/ELB.

One alternative to the configuration research route is to use an ALB instead of an ELB. They're like ELBs but they support WebSockets, and so don't require Shiny to fall back to any non-WebSocket transport. The one trick to them is that if you're running a cluster of Shiny Servers or Connects behind your ALB, you must configure sticky sessions.

[shimberger]

@alandipert Thanks so much for getting back to me so quickly.

You are indeed correct: I am using the Application Load Balancer (ALB). Sorry I wasn't precise. The symptom is that after the configured idle timeout (https://s3-eu-west-1.amazonaws.com/captured-krxvuizy1557lsmzs8mvzdj4/bp7p6-20180626-26091742.png) the screen goes gray.

This seems to be due to the fact that if the user is idle no traffic is flowing over the TCP connection of the WebSocket. I've reproduced this by measuring the time of when the screen greyes out and comparing it to the ALB timeout. I also changed the ALB timeout and the behaviour changed accordingly. I am not sure how sticky sessions could help but I am willing to give it a try if you still think that's what it is.

Just for the record: I am not using shiny server or connect. Just docker and shiny:runApp behind an Apache reverse proxy.

Thanks so much again.

[wch]

I've also seen this behavior on rstudio.cloud, which uses an ALB which is configured for a 5 minute timeout. If a user starts an app there and lets it sit without any interaction, the session closes after 5 minutes (although the app keeps running).

[dpmccabe]

Here's a simple solution to increment a counter every 10 seconds.

JS

var socket_timeout_interval;
var n = 0;

$(document).on('shiny:connected', function(event) {
  socket_timeout_interval = setInterval(function() {
    Shiny.onInputChange('alive_count', n++)
  }, 10000);
});

$(document).on('shiny:disconnected', function(event) {
  clearInterval(socket_timeout_interval)
});

CSS

#keep_alive {
  visibility: hidden;
}

server.R

output$keep_alive <- renderText({
  req(input$alive_count)
  input$alive_count
})

ui.R

textOutput("keep_alive")

[falaki]

We are encountering this issue as well. The idle timeout for the websocket connection in Shiny cannot be effectively controlled by Shiny unless it we implement a heartbeat mechanism.

Although there is an internal idle timeout that is said to be configurable in RStudio Connect, the websocket could still be terminated by other components along the path such as cloud load balancers, whichever has a shorter idle timeout. This is even more severe when not using RStudio Connect.

We encountered this issue with load balancers in AWS and Azure. The load balancer timeout could be as short as 1 minute, which could harm the user experience of shiny even for the development setting.

Shiny can consciously control or prevent the timeout end-to-end by introducing a heartbeat mechanism. See the websocket RFC section on ping/pong messages.
With this mechanism, Shiny would be more robust in the cloud environments and open up opportunities for more use cases.

[ciaranevans]

@shimberger Did you ever get around this? I think I'm experiencing the same thing.

[iqis]

my solution, if I understood the problem correctly:

timer <- reactiveTimer(1000 * 60 * 5) # time unit in milliseconds
        observe({
            timer()
            {{"here insert your code to submit a message through websocket client"}}
        })