Persistently storing and retrieving data from R Shiny Apps

An advanced guide for file handling in R Shiny (including PDFs)

Photo by Mr Cup / Fabien Barral on Unsplash

My last post intrigued quite a lot of people because of the novel and innovative nature of my last R Shiny app. This use case of Shiny was mostly unheard of, as primarily this is not what R Shiny was designed for. However, due to some personal and professional reasons, I decided to undertake the project of “Creating a centralized platform for my university exam resources” in R Shiny and here in this article, I’ll share the solution to the biggest hurdle you might face in file handling using R Shiny.

The Problem: Persistent Data in R Shiny

Whenever you are dealing with persistent data storage in R Shiny apps, you are bound to come across articles by Dean Attali, especially this article. This article, covers a wide variety of possible scenarios for you. I, however, was left wanting for more, as it still did not align with my requirements.

What I wanted was the functionality for users to upload any number of files and then for every other users to be able to download those files.

When you work with R Shiny, you would across the functions fileInput() and the downloadHandler(). These functions can only help you to work with temporary files (files that will remain as long as the “session” is not changed). To persistently store the data, I tried exporting the “temporary” uploaded file to a “DropBox” profile with the help of “rdrop2”, but this process was painfully slow. Also since I was dealing with PDF files, connecting MongoDB servers or MySQL servers was not a viable option.

The Solution: Local Storage

This whole ordeal made one thing painfully clear for me. Remote storage could not be used for my project and local storage was the way to go.

So, if you look into the available resources on the internet, the closest you’d come to accomplishing this is by following Dean’s example here. But, here again, he is simply using the function write.csv() and providing the local storage path name in the path parameter of the function. This still leaves us to figure out how do you “write” a PDF file to a specific location (www folder) in your local storage.

I searched online for R packages that deal with PDFs and I came across the packages “pdftools” and “qpdf” . Since “qpdf” was a requirement of the “pdftools” package, I decided to only install the “pdftools” package. Unfortunately, this function too did not provide us the functionality to “write a PDF” onto a given file path. The functions “pdftools” provided are listed below.

Description of the “pdftools” R package

The functions provided by the “qpdf” package are as follows.

Description of “qpdf” R package

Amongst these functions, what caught my eye was the function pdf_subset() , because it had a parameter called “output”. So, it could potentially mean to store a subset of a particular PDF file in the specified file path by means of the output parameter. The “pages” parameter was specified using the above mentioned pdf_info() function (pdf_info$pages to be more specific).

And, VOILA!!! This little experiment of mine worked and I attained the functionality that I wanted to. Here’s a code snippet from my actual deployed app highlighting how I managed the upload part in my app.

https://gist.github.com/hinduBale/5356a228372b3ec8fe08ea04c1802b71

output$upload_questionSet_button <- renderUI({                           validate(                             
need(input$display_name_ques != "", "Please enter the display name of the uploader")                           
 )                           
fileInput( inputId = "multi_sol_id",                                      label = "Please upload the question set available with you :) (PDFs only)",                                                       multiple = TRUE,                                                accept = c(".pdf"),                                        buttonLabel = "Upload Papers")       
})  
                                                observeEvent(input$multi_sol_id, {                           fileName_set <- sprintf("www/%s/questionSet_%s_%s.pdf", input$sem_select_mul,input$exam_select_mul, input$display_name_ques)                           pages_set <- pdf_info(input$multi_sol_id$datapath)$pages                           pdf_subset(input$multi_sol_id$datapath, 
           pages = 1:pages_set, 
           output = fileName_set)
})

Now, after the files were being uploaded successfully, I needed a way to actually be able to download resources from my local storage using R Shiny. For this too, I searched far and wide, read the official docs, but to no avail. After tinkering around with all available options and a voracious reading for relevant materials over the internet, I got a little hint from this Stack Overflow answer. Therefore, I used the file.copy() function and converted the “filename” and “content” parameters to function to accomplish my task.

Here’s a code snippet from my actual deployed app that showcases how I handled the download part of my app.

https://gist.github.com/hinduBale/eddf858962865f43f4ac152116dc84e3

output$download_single_qp_button <- renderUI({
    downloadBttn(
      outputId = "qp_down_sin", 
      label = "Download Question Paper :)", 
      style = "float", 
      color = "warning", 
      size = "lg", 
      no_outline = FALSE)                
  })
  
  output$qp_down_sin <- downloadHandler(
    filename <- function(){
      search_ques_fileName <- sprintf("www/%s/%s",
                                      input$sem_select_solution,
                                      input$avail_paper)
      return (search_ques_fileName)
    },
    content <- function(file) {
      file.copy(filename(), file)
    },
    contentType = "application/pdf"
  )

Conclusion:

So, we have actually been able to accomplish what we set out to do. It sure is euphoric when you solve a problem, isn’t it!!

Photo by Brooke Cagle on Unsplash

I seriously felt like punching my screen many times when I was solving this problem, hence I wrote this article so that it may save a precious life or at least a precious monitor 😜

One thing to note here, is that, at the point of writing, https://www.shinyapps.io/ does not provide the facility of a local storage (at least, not in their free tier). So, you’ll have to set up your own R server on the cloud service provider of your choice if you want to reap the benefits of local storage.

You could also buy me a coffee to support my work.

Thank You and Godspeed.