Shiny: downloadHandler() fails to offer file up for download in containerized app
I have a shiny app that pulls data from Google Analytics based on user input, passes that data to an .Rmd file, knits to a beamer presentation .pdf, then offers that file for download. This app works without error locally.
I have containerized that shiny app with docker file
FROM rocker/verse:3.5.1
LABEL Steven H. Ranney "[email protected]"
## Add shiny capabilities to container
RUN export ADD=shiny && bash /etc/cont-init.d/add
## Update and install
RUN tlmgr update --self
RUN tlmgr install beamer translator
## Add R packages
RUN R -e "install.packages(c('shiny', 'googleAuthR', 'dplyr', 'googleAnalyticsR', 'knitr', 'rmarkdown', 'jsonlite', 'scales', 'ggplot2', 'reshape2', 'Cairo', 'tinytex'), repos = 'https://cran.rstudio.com/')"
#Copy app dir and theme dirs to their respective locations
COPY app /srv/shiny-server/
COPY app/report/themes/SwCustom /opt/TinyTeX/texmf-dist/tex/latex/beamer/
#Force texlive to find my custom beamer thems
RUN texhash
EXPOSE 3838
## Add shiny-server information
COPY shiny-server.sh /usr/bin/shiny-server.sh
COPY shiny-customized.config /etc/shiny-server/shiny-server.conf
## Add dos2unix to eliminate Win-style line-endings and run
RUN apt-get update && apt-get install -y dos2unix
RUN dos2unix /usr/bin/shiny-server.sh && apt-get --purge remove -y dos2unix && rm -rf /var/lib/apt/lists/*
CMD ["/usr/bin/shiny-server.sh"]
and with server.R
library(shiny)
library(googleAuthR)
library(googleAnalyticsR)
library(rmarkdown)
library(jsonlite)
clients <- read.csv("data/client_keys_2018_07_10.csv",
fileEncoding = "UTF-8-BOM",
header = TRUE)
client_id <- fromJSON("client_id.json")
options(googleAuthR.client_id = client_id$installed$client_id)
options(googleAuthR.client_secret = client_id$installed$client_secret)
# Define server logic required to pull and process data
shinyServer(function(input, output) {
#Establish the Google Service account as the Server to Server OAuth
gar_auth_service(
json_file = "ranney-sw-test-2018-06-22-ccfcdd77f262.json",
scope = "https://www.googleapis.com/auth/analytics"
)
# Determine which client code to use from the input page
client_code <-
reactive({(clients %>%
filter(client_name == input$client %>% as.character()) %>%
select(client_key) %>%
unique())$client_key})
client <- reactive({dim_filter("customVarValue1", operator = "EXACT", client_code() %>% as.character(), not = FALSE)})
## construct filter objects
fc <- reactive({filter_clause_ga4(list(client()), operator = "AND")})
dataInput <-
reactive({
withProgress(min = 0, max = 1, {
incProgress(message = "Pulling data from Google Analytics servers",
detail = "This may take a while...", amount = .1)
google_analytics(viewId = "83057204", # Corresponds to SWP V5, filtered view ,
date_range = c(input$start_date, input$end_date),
dimensions = c("customVarValue1", "city", "dateHour"),
metrics = c("users", "newUsers", "entrances", "sessions", "sessionsPerUser",
"avgSessionDuration", "uniquePageViews"),
dim_filters = fc(),
max = -1) %>%
rename(client_key = customVarValue1) %>%
mutate(client_key = client_key %>% as.numeric(),
date = substr(dateHour, 1, 8) %>% as.Date(format = "%Y%m%d"),
hour = substr(dateHour, 9, 10) %>% as.numeric(),
city = ifelse(city == "(not set)", "NA", city),
avg_session_duration_min = avgSessionDuration/60,
day_of_week = strftime(date, "%A"),
day_of_week = factor(day_of_week, levels = c("Monday", "Tuesday", "Wednesday",
"Thursday", "Friday", "Saturday",
"Sunday")),
existing_users = users - newUsers)
})
})
output$generate <- downloadHandler(
# This function returns a string which tells the client
# browser what name to use when saving the file.
filename = function() {
paste0(input$client, "_", Sys.Date(), ".pdf") %>%
gsub(" ", "_", .)
},
# This function should write data to a file given to it by
# the argument 'file'.
content = function(file) {
withProgress(min = 0, max = 1, {
incProgress(message = "Processing data into report",
detail = "This may take a while...", amount = .1)
params <- list(data = dataInput(),
client = input$client)
# Write to a file specified by the 'file' argument
render("report/GA_report.Rmd",
output_format = "all",
output_file = file,
params = params,
envir = new.env())
# clean = FALSE,
# quiet = FALSE)
})
}
)
})
The app does not return the file for download. The download dialog box opens but the filename defaults to generate (the name of the download button in my ui.R):
The app is creating the .pdf files from the .Rmd as is evidenced by the shiny logs:
Listening on http://127.0.0.1:36325
2018-08-08 16:07:55> No scopes have been set, set them via
options(googleAuthR.scopes.selected) -
no authentication attempted.
Attaching package: ‘dplyr’
The following objects are masked from ‘package:stats’:
filter, lag
The following objects are masked from ‘package:base’:
intersect, setdiff, setequal, union
2018-08-08 16:07:55> Default Google Project for googleAnalyticsR is now set.
This is shared with all googleAnalyticsR users.
If making a lot of API calls, please:
1) create your own Google Project at https://console.developers.google.com
2) Activate the Google Analytics Reporting API
3) set options(googleAuthR.client_id) and options(googleAuthR.client_secret)
4) Reload the package.
2018-08-08 16:07:55> Set API cache
2018-08-08 16:07:55> No environment argument found, looked in GA_AUTH_FILE
Attaching package: ‘jsonlite’
The following object is masked from ‘package:shiny’:
validate
2018-08-08 16:08:05> Downloaded [585] rows from a total of [585].
processing file: GA_report.Rmd
`geom_smooth()` using method = 'loess' and formula 'y ~ x'
`geom_smooth()` using method = 'loess' and formula 'y ~ x'
`geom_smooth()` using method = 'loess' and formula 'y ~ x'
`geom_smooth()` using method = 'loess' and formula 'y ~ x'
output file: GA_report.knit.md
Output created: /tmp/Rtmpxswbsu/file181f90784c.pdf
2018-08-08 16:09:37> Downloaded [536] rows from a total of [536].
processing file: GA_report.Rmd
`geom_smooth()` using method = 'loess' and formula 'y ~ x'
`geom_smooth()` using method = 'loess' and formula 'y ~ x'
`geom_smooth()` using method = 'loess' and formula 'y ~ x'
`geom_smooth()` using method = 'loess' and formula 'y ~ x'
output file: GA_report.knit.md
Output created: /tmp/Rtmpxswbsu/file1847e50e9f.pdf
What could be causing the error of the app to convert the .Rmd to a .pdf but not make it available for download?
The .Rmd is rather big and takes a while to generate. Is it possible that the downloadHandler() function is presenting the download dialog before the .pdf is generated? I tried to recreate this problem with another dockerized shiny app with non-proprietary data but could not. In that second app, the .Rmd file was much smaller. This is the only difference between the two apps. Does the time required to generate a report affect what is presented by downloadHandler()?
stevenranney
All 5 comments
This could be a timing issue, based on this:
https://github.com/rstudio/shiny-server/issues/197
I'll adjust my ui.R and server.R to separate report generation from report download and try again.
stevenranney
on 8 Aug 2018
Once I refactored my app to keep the render() out of the downloadHandler(), everything works as I expect it to.
I had no idea that there was a hard-coded time-limit in downloadHandler().
stevenranney
on 8 Aug 2018
@stevenranney, can you please show code how to separate render() and downloadHandler()? I got the same problem
kablag
on 30 Aug 2018
Sure. You just have to separate the render() from the downloadHandler(). Here's a basic ui.R file that has a "generate report" button and upon completion, shows a "download report" button:
library(shiny)
library(googleAuthR)
library(dplyr)
# Define UI
shinyUI(fluidPage(
# Application title
titlePanel("TITLE"),
sidebarLayout(
sidebarPanel(
selectInput("client",
label = h4("Client"),
choices = [my own data] %>% .$client_name %>% unique()),
dateInput("start_date", label = h4("Start date"), value = "2017-06-30"),
dateInput("end_date", label = h4("End date")),
actionButton("generate", "Generate Report", icon = icon("file"), # This is the only button that shows up when the app is loaded
style = "color: #fff; background-color: #337ab7; border-color: #2e6da4"),
br(),
br(),
conditionalPanel(condition = "output.reportbuilt", # This button appears after the report has been generated and is ready for download.
downloadButton("download", "Download Report",
style = "color: #fff; background-color: #337ab7; border-color: #2e6da4"))
),
# Show a plot of the generated distribution
mainPanel()
)
))
Then, on the server side, you just need to create a a reactive value that can store a filepath short-term, create an event that, upon clicking the generate button in the UI render()s the report, then serve the report for download normally.
library(shiny)
library(rmarkdown)
library(jsonlite)
# Define server logic required to pull and process data
shinyServer(function(input, output) {
report <- reactiveValues(filepath = NULL) #This creates a short-term storage location for a filepath
observeEvent(input$generate, {
progress <- shiny::Progress$new()
# Make sure it closes when we exit this reactive, even if there's an error
on.exit(progress$close())
progress$set(message = "Gathering data and building report.",
detail = "This may take a while. This window will disappear
when the report is ready.", value = 1)
client_code <-
(clients %>%
filter(client_name == input$client %>% as.character()) %>%
select(client_key) %>%
unique())$client_key
client <- dim_filter("customVarValue1", operator = "EXACT", client_code %>% as.character(), not = FALSE)
## construct filter objects
fc <- filter_clause_ga4(list(client), operator = "AND")
raw_GA_data <- #Goes and gets data based upon user choices
[data processing]
if(is.null(raw_GA_data))
showNotification(paste("No data exists for this client/time
period combination. Please select another client."),
type = "error",
duration = NULL,
closeButton = FALSE,
action = a(href = "javascript:location.reload();", "Reload page")
)
dataInput <- # data processing step
raw_GA_data %>% processing...
params <- list(data = dataInput,
client = input$client)
tmp_file <- paste0(tempfile(), ".pdf") #Creating the temp where the .pdf is going to be stored
render("report/GA_report.Rmd",
output_format = "all",
output_file = tmp_file,
params = params,
envir = new.env())
report$filepath <- tmp_file #Assigning in the temp file where the .pdf is located to the reactive file created above
})
# Hide download button until report is generated
output$reportbuilt <- reactive({
return(!is.null(report$filepath))
})
outputOptions(output, 'reportbuilt', suspendWhenHidden= FALSE)
#Download report
output$download <- downloadHandler(
# This function returns a string which tells the client
# browser what name to use when saving the file.
filename = function() {
paste0(input$client, "_", Sys.Date(), ".pdf") %>%
gsub(" ", "_", .)
},
# This function should write data to a file given to it by
# the argument 'file'.
content = function(file) {
file.copy(report$filepath, file)
}
)
})
There's probably a much better way to streamline this process but this is what worked for me.
stevenranney
on 30 Aug 2018
Hi @stevenranney,
I am referencing to your solution as I run into similar problem where i wanted to export the plots from render functions but the pdf is returning 0 pages. Would it be possible to share how your GA_report.Rmd codes look like? I have no experience in rmarkdown & would be interesting if i could use your approach to resolve my problem.
lambermont
on 24 May 2020
Related issues
daattali
·
3Comments
hlherrera
·
5Comments
daattali
·
4Comments
daattali
·
5Comments
mdec
·
5Comments
Most helpful comment
Sure. You just have to separate the
render()from thedownloadHandler(). Here's a basicui.Rfile that has a "generate report" button and upon completion, shows a "download report" button:Then, on the server side, you just need to create a a reactive value that can store a filepath short-term, create an event that, upon clicking the
generatebutton in theUIrender()s the report, then serve the report for download normally.There's probably a much better way to streamline this process but this is what worked for me.