Max Kuhn - Serverless Quarto
Serverless Quarto - Max Kuhn Resources mentioned in the presentation: - Slides: https://topepo.github.io/2023-r-pharma - Example: https://topepo.github.io/shinylive-in-book-test Bio: Max Kuhn is a software engineer at Posit (née RStudio). He is working on improving R's modeling capabilities and maintaining about 30 packages, including caret. He was a Senior Director of Nonclinical Statistics at Pfizer and had been applying models in the pharmaceutical and diagnostic industries for over 18 years. Max has a Ph.D. in Biostatistics. He, and Kjell Johnson, wrote the book Applied Predictive Modeling, which won the Ziegel award from the American Statistical Association. Their second book, Feature Engineering and Selection, was published in 2019, and his book Tidy Models with R, was published in 2022. Presented at the 2023 R/Pharma Conference (October 25, 2023)
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
All right, thanks. Yeah, that describes it pretty well. Using Shiny without a Shiny server.
Quarto is a document preparation system. If you've never heard of it, it's sort of like the successor to R Markdown that sort of integrates blogs and presentations and documents and books. So I'll mostly be focused on how to use Shiny inside of Quarto. So I think the links will be in Slack, but here is the link to the slides. And then there'll be a link to a separate repo that has the example book. All right, so shinylive is both an R and Python.
For R, there's a shinylive Quarto extension, as well as an R CRAN package that you can use. And what it does is it uses the WebAssembly version of R. And that's like a precompiled version of R that could be used in your browser and interfaced using JavaScript. So basically, we take R and packages that you're going to use in your Shiny app, we bundle them together in a big binary object, and then we use JavaScript in the browser to access that. So it's all done on your browser. It's all done on your iPhone or iPad or whatever you're using to access the website.
There's links here for the Quarto extension in the R package.
Motivation and setup
Oh, I should say that my goal here initially was, you write books on machine learning and things like that. We want to show, for example, like how model complexity affects, let's say, classification models. And so what we'd like to, instead of having a ggplot with a bunch of facets, we'd like to have some shiny sliders that we can say, like, increase complexity and see what the results are in a plot. So that's really where I would like to go with this. So what you do is, when you're running your Quarto document, you basically create a code chunk that instead of Python has the type ShinyLive-R. And then all your app and UI stuff goes within that chunk.
Now, the one thing that you do have to do is you have to give it an extra option, standalone equal true. And that's just required to set everything up for ShinyLive to work. Another thing you have to do is in your main Quarto file, in the YAML, you have to add a filter for ShinyLive. And from there, it's kind of off to the races.
Now, one thing that's, and I should say, a lot of this is currently in flux. Joe Chang did a great talk at Posit Conference about this a few months ago. And I spent, like, a week trying to figure it out. And even since then, the repo that I have, I have a pull request in this morning to make some updates based on some feedback they gave me yesterday. So it's very much a work in progress. And so you might find things that are inconsistent when you go to look at this with what I'm showing on my slides, because we have usually, like, a better interface of doing some of these things.
So it's very much a work in progress. And so you might find things that are inconsistent when you go to look at this with what I'm showing on my slides, because we have usually, like, a better interface of doing some of these things.
Packages and data access
So anyway, if you're going to download R packages into the browser, it needs to know what packages you want to use. And ShinyLive uses Rn, so you can either just have library calls inside of your Shiny chunk, or also write a description file if you have something more complicated you want to do. Also, there's a nice link to some of the discussions about this and where this is going.
Now, the biggest thing to realize if you're going to use ShinyLive in the browser is that it's all done through JavaScript. And what that means is you get all the benefits and all of the security benefits of JavaScript, but the security benefits sometimes constrain you, because JavaScript doesn't want you just executing anything or giving you access to everything automatically. So one issue is, what do you do about data?
So I, for my example, precomputed, like, grids of predictions so I could show class boundaries, and I wanted to include that in the Shiny app, so I had to find a way to get that in. And so there's a couple ways you can go about that. If you look at that GitHub issue that's highlighted there, they're looking at adding, like, local folder support, where you put all the things you want to access in your Shiny app into one specific place that's sort of qualified as saying this is, like, a safe place to get these objects. Other than that, you should probably use something like download.file to get things from a remote resource. And in my particular example, I put the R data file on GitHub and used the GitHub raw data interface, like, URL interface to do that.
Download file with JavaScript is a little bit difficult, because you have to make sure you do the request in a way that JavaScript wants you to, which is based on, like, headers of the documents and honestly stuff that I don't understand. So what I would do if I were you is look at the example repo and see how I did it and also look at this issue in GitHub to see what's been described.
All right. So just so you know, you know, you're executing Shiny and your data in a web browser, which means that everything that you do to make that app goes to the client's machine. Everything's run locally on your computer or phone or whatever. So just be careful not to include credentials or other sense of data or patient data. This kind of goes without saying, but it's easy to sort of lose track of how this is different from standard Shiny.
Live demo
All right. So to get to the meat of it, here's the repo that actually has the book in it. I'm going to click on this. And there's a lot of details on how to create this in Quarto. The second page here has a lot of the things you need to do, like the example application and things like that.
The thing that when you come down to, here's the actual Shiny app. And so right now what it's doing is it's downloading the precompiled R and the packages that I need. And it's going to take, like, in my example this morning, it's like about 30 or 40 seconds to start. And this is going to get faster because we're looking at ways to pre-bundle packages and things like that.
Now, if you want to see the end result, I opened it earlier from the cheat. And here's what it looks like. Is I have some data with two classes and I have a logistic regression using some spline features. So if I, like, increase it from three to six splines, like, how complex does this get? And this is the kind of example that I want to show using Shiny and Quarto. So this is how you do it, basically.
Here's the end result. Let's see. Has it started by now? Yeah. So the original link has already loaded everything. So this repo is probably the one you want to look at.
Again, the links are in Slack. I will have to update it periodically as we improve things. But there's details on how to reference this actually as a figure, like a static figure. How should you, like, for example, access your data. So, again, I use download.file with the raw GitHub content access point. So all the details are here. And it's probably a good place to stop and take questions if we have them. Oh, most importantly, I wanted to thank where are the slides at? George and Joe and Winston and everybody who made all this happen.
It's an amazing kind of feat, to be honest with you. As Joe said in his talk, it's kind of hard to believe this is actually happening, but it is.
it's kind of hard to believe this is actually happening, but it is.
Q&A
All right. Awesome. Excellent. Very interesting stuff to add to our toolbox. Very, very great. Some questions. We have a little bit of time. Bruno is asking what would the baseline file size of a simple serverless Shiny app? That's a great question. I don't particularly know.
So one thing I did last night is since you used that standalone equal true, I was like, well, is this downloading some big file with a big chunk of data? I think it does that on the fly. So I couldn't tell you right now because it happens at the time at which you render the slides. So you're not, like, parking that in GitHub in, like, a 300 megabyte file or whatever it is. It's doing it whenever you start things up. I believe that the intent is for us to have maybe pre-bundled collections of packages and things like that so you don't have to wait for that startup time or just load it locally. Again, I'm not on the Shiny team, but that's my understanding of maybe where they want to go. But right now it's hard for me to say because it all happens sort of on the fly without having at least access that I can find to it.
Right. So maybe we have time for one more question from Nate. Since the package have to be pre-compiled, you can't use with internal packages. Is that correct? I believe that's true right now. But again, this is something that I know the Shiny devs have discussed. I think that maybe if you go to the shinylive page, there might already be instructions on how you can pre-compile your local package or whatever to work with this. I know I've seen that.
I think it's public. But I haven't done it. So I would look at the shinylive or maybe the web R page would probably tell you how to do that. Awesome. Excellent. Great. Thank you so much, Max. We can keep the discussion going in the chat. I'm sure Max will be happy to answer some more questions as we move along with it.
