Shiny: Possibly Memory Leak?

[Silentdevildoll]

I was suggested to write an issue on stackoverflow: https://stackoverflow.com/questions/53402267/shiny-memory-leak-r?noredirect=1#comment93680044_53402267

To put it simply, this code, using pryr and shiny, seems to just build up memory. Is this an issue? After an hour the code for me goes up about 50-100 mbs:

library(shiny)
library(pryr)

ui<-fluidPage(
  textOutput("Memory"))

server <- function(input, output, session) {
  memUsed <- function() {paste0(round(mem_used()/1000000,3)," mb")}

  output$"Memory"<-renderText({
    invalidateLater(10)
    memUsed()
  })
}

shinyApp(ui, server)

[jcheng5]

I'd be a bit surprised, but it's conceivable; we periodically (manually) test for memory leaks but nothing like invalidateLater(10) for an hour. We can look into it.

Does your actual app run invalidateLater(10), though? You know that's 10 milliseconds, right?

[jcheng5]

I'd also be curious--if you stop the app, and then run gc() a couple of times, does the memory drop?

Finally, what versions of R, shiny, and httpuv are you using? (or just devtools::session_info())

Thanks.

[Silentdevildoll]

In my own app, I actually just use an action button to print memory every time it is pressed. From what I can tell it seems to be about the same. For the invalidate, originally I was doing it every second (1000) but I was under the impression it would illustrate faster with 10 ms. Either way it seems to have the same result, steadily building.

I did a mem_used() before running, 63.9 mb. I let the program run until it showed 80 mb, which was about 5 minutes. Mem_used immediately after closing was 66.3 mb. I ran gc() three times, then mem_used went up to 76.2 mb surprisingly.

Session info ---------------------------------------
setting value
version R version 3.4.2 (2017-09-28)
system x86_64, mingw32
ui RStudio (1.1.383)
language (EN)
collate English_United States.1252
tz America/Los_Angeles
date 2018-11-20

Packages -------------------------------------------
package * version date
base * 3.4.2 2017-09-28
compiler 3.4.2 2017-09-28
datasets * 3.4.2 2017-09-28
devtools 1.13.6 2018-06-27
digest 0.6.15 2018-01-28
graphics * 3.4.2 2017-09-28
grDevices * 3.4.2 2017-09-28
htmltools 0.3.6 2017-04-28
httpuv 1.4.5 2018-07-19
later 0.7.3 2018-06-08
magrittr 1.5 2014-11-22
memoise 1.1.0 2017-04-21
methods * 3.4.2 2017-09-28
mime 0.5 2016-07-07
promises 1.0.1 2018-04-13
R6 2.2.2 2017-06-17
Rcpp 0.12.18 2018-07-23
shiny * 1.1.0 2018-05-17
stats * 3.4.2 2017-09-28
tools 3.4.2 2017-09-28
utils * 3.4.2 2017-09-28
withr 2.1.2 2018-04-02
xtable 1.8-2 2016-02-05
yaml 2.1.19 2018-05-01
source
local
local
local
CRAN (R 3.4.4)
CRAN (R 3.4.3)
local
local
CRAN (R 3.4.2)
CRAN (R 3.4.4)
CRAN (R 3.4.4)
CRAN (R 3.4.2)
CRAN (R 3.4.3)
local
CRAN (R 3.4.1)
CRAN (R 3.4.4)
CRAN (R 3.4.2)
CRAN (R 3.4.2)
CRAN (R 3.4.4)
local
local
local
Github (jimhester/withr@79d7b0d)
CRAN (R 3.4.3)
CRAN (R 3.4.4)

[jchrom]

Is this expected to build up memory? It goes up pretty fast.

library("shiny")

ui <- fluidPage(uiOutput("button"), uiOutput("dtout"))

server <- function(input, output) {

  output$button = renderUI({ actionButton("go", "Go") })

  d = ggplot2::diamonds

  output$dt = DT::renderDT({
    input$go
    rbind(d, d, d, d, d)
  })

  output$dtout = renderUI({ DT::DTOutput("dt") })
}

shinyApp(ui,server)

[shrektan]

@jchrom No, it's because the garbage collector hasn't collected the unused memories. Try the code below, because pryr::mem_used() will call gc(), you can find the memory is basically not changed even after multiple clicking.

library("shiny")

ui <- fluidPage(verbatimTextOutput('mem'), uiOutput("button"), uiOutput("dtout"))

server <- function(input, output) {

  output$button = renderUI({ actionButton("go", "Go") })

  d = ggplot2::diamonds

  output$dt = DT::renderDT({
    input$go
    rbind(d, d, d, d, d)
  })

  output$dtout = renderUI({ DT::DTOutput("dt") })
  output$mem = renderPrint({input$go; pryr::mem_used()})
}

shinyApp(ui,server)

[jchrom]

Thanks for the tip.

The pryr output stays on a fixed value. But when I check the memory usage via the browser dev tools, it is climbing up like before, no change. This is tested on current Chrome and Firefox, running the app locally on Ubuntu 18.04 R 3.4.4 and Shiny 1.2

[shrektan]

You are watching the memory of the browser, not the server. The browser will collect the memory by itself and should have nothing to do with shiny, IMO.

[jchrom]

Aha! Looks like I have some reading to do. In case you are still entertained providing useful insights for me, can you tell me whether increasing memory build-up I can see in the browser is considered an issue in any sense? Or should I just ignore that in favour of whatever pryr is telling me?

[ramindehghanpoor]

@Silentdevildoll @jcheng5 Did you find any answer? I have the same problem and need help.

[wch]

@ramindehghanpoor Have you installed the development version of shiny from Github? We recently merged in some fixes for memory leaks.

[ramindehghanpoor]

@wch Thanks for your answer. Yes, I reinstalled the development version of shiny from Github and still have the memory leak problem. Here is a simple code with this problem:

library(shiny)
library(pryr)

ui <- fluidPage(
  plotOutput("hist1"),
  verbatimTextOutput("Memory")
  )

server <- function(input, output, session) {
  output$hist1 <- renderPlot({

    #invalidate every 1 seconds
    invalidateLater(1000)

    plot(c(1:100))

    #show the memory usage
    output$Memory <- renderText({paste0("Memory usage: ", mem_used())})
  })
}

shinyApp(ui,server)

[ramindehghanpoor]

@wch @jcheng5 It's really important for our company. Every day we are getting out of memory because of this problem. Do you have any solution? Do you think Shiny will fix this bug soon?

[Silentdevildoll]

Just as a comment on my end, I know the majority of my problem was either with my code itself, or perhaps how I was hosting it? When I switched to hosting it using task scheduler instead of through R itself, as well as rewrote the majority of my code using things like shiny modules, I was able to really minimize the memory increase. When I originally wrote this, I was increasing on the scale of several hundreds of megabytes a day. Now, in a two week period I'm increasing about 12 mb. So I'm just saying that I still think this issue may exist, but for my purposes it is at most only causing a small increase in memory.

[wch]

It appears that there may be a leak with invalidateLater because a callback is registered each time invalidateLater is run, but it is never cleared (until the session ends):

https://github.com/rstudio/shiny/blob/4c0af8b1/R/reactives.R#L1630