Shiny New Tools for Scaling your Shiny Apps - posit::conf(2023)
Presented by Joe Kirincic So you have a Shiny app your org loves, but as adoption grows, performance starts getting sluggish. Profiling reveals your cool interactive plots are the culprit. What can you do to make things snappy again? We can increase the number of app instances, sure, but suppose that isn't an option for us. Another approach is to shift the plotting work from the server onto the client. In this talk, we'll learn how to leverage two Javascript projects, DuckDB-WASM and Observable's Plot.js, in our Shiny app to create fast, flexible interactive visualizations in the browser without burdening our app's server function. The end result is an app that can scale to more users without needing to increase compute resources. Presented at Posit Conference, between Sept 19-20 2023, Learn more at posit.co/conference. -------------------------- Talk Track: The future is Shiny. Session Code: TALK-1088
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
So, thank you all for having me out today and joining me for my talk. Again, my name is still Joe Kurinsic, and I'm here to talk to you all about scaling your Shiny apps and some cool tools to do that.
But before we get into that, I'm going to talk about something a little different, which is Rec League Sports. In these low-stakes competitive arenas, there's a number of player archetypes that emerge. But one that shows up often is the overextender. This person is putting it all on the line on their team. They're doing everything, everywhere, all at once, and with an unrelenting fervor to boot.
We love this player. We appreciate what they do. But what becomes apparent very quickly is that they reach a point where they're doing too much, and their performance just starts to get weird. It starts to lag.
And truth be told, our Shiny apps are very much the same way. You know, the process goes, you develop a prototype, you share it with a particular team. Team loves it. Word starts getting around. Other teams begin to leverage your app as well. Adoption continues to increase. But you reach a certain critical mass where your Shiny app is serving a lot of these teams But the performance starts to lag. You know, the complaints start to roll in. My plots are lagging. Or my report's taking too long to run. Or you just get that spinning wheel of death that seems to be taking just a little too long.
The problem with scaling Shiny
So this talk, okay, this talk is going to be a story about two things. One is going to be about a toy app that struggled to scale to a large number of users. And then we're also going to talk about how we use JavaScript to augment Shiny to overcome that struggle.
So this is the app that we start with. It's a very simple Shiny application with two interactive hex bin plots. Simple easy and nice. So with an app like this, you know, what seems to be the issue? The issue happens when we go from one concurrent user, which is me operating this app locally, to when we have 100 concurrent users accessing this app at the same time.
And that problem is underscored by this plot that I'm going to show you all right here. Now for some of you that aren't familiar with kind of like load testing tools for Shiny, this plot may be a little bit opaque. So let's try to break it down a little bit. So this is what's known as a session duration plot. And it's showing 100 simulated users using the Shiny app.
The ticks along the Y axis are basically representing your simulated users as they go through the workflow on the previous slide. And then each of these segments that are stretching across the X axis, that's the amount of time that it took a particular step in that user's workflow. And then we have, it's a, sorry, it might be skinny for those in the back, but there's a red line here that is kind of a reference point that indicates how long the total workflow took for a single concurrent user. And here, for this workflow, is about 93 seconds.
So there's two things that stand out in this plot. One is that we're noticing some inconsistency in when our users are finishing the workflow. From a UX perspective, the same workflow should take people the same amount of time if they're accessing the app at the same time. That's just fair.
Another thing to notice is that the amount of time that it's taking to go through this workflow is, has gotten longer. And in the worst case, we're seeing that some users are finishing close to 150 seconds, as opposed to the 93 seconds for the one user. There's no bueno.
So we have this problem, why is it happening? To understand why it's happening, we want to quickly just run through some basics about how Shiny works. Whenever we update one of the dropdowns in those hex bin plots, the browser is going to send a signal over to R and say, hey, can you please generate another plot for me? And then R is going to be like, sure, buddy, I got you. And it's going to generate another PNG, which then the browser goes to render. So we have this nice bidirectional communication going.
And that's nice when you have just one user, like when you're developing locally and whatnot. But this becomes a problem when you have many browsers connecting to your app simultaneously. Most of the time, when you have your Shiny app deployed, there's ultimately a single R process that's underlying that app. And when all of these browsers are connecting to it, that single R process is doing the brunt of the work to serve up your plots, tables, et cetera.
Now, R is a very fast language. But R is still a single-threaded language. And so because of that, every Shiny app out of the box is going to reach a certain critical point where the R process becomes saturated. And it develops a backlog of requests that it has to mow through. And that backlog is what causes that increase in the session duration that we saw in the plot earlier.
And so because of that, every Shiny app out of the box is going to reach a certain critical point where the R process becomes saturated. And it develops a backlog of requests that it has to mow through.
Using JavaScript to fix the problem
So that's, you know, we have a sense for what the problem is. So what are we going to do about this? Now, there's been plenty of talks about scaling Shiny. And there's a lot of good strategies that have come from those. We can cache different assets. That's a good idea. We should be doing that, if we're not already. There's also asynchronous operations, you know, writing out files, like large IOE-type stuff. But that's not what we're here to talk about. We're here to talk about something a little weirder. We're going to talk about using JavaScript to fix this problem.
So what's the, like, use JavaScript, what are we going to do? The big idea here is that we can scale our Shiny applications by taking work from that R process, that R session, and moving it into the browser.
If we think back to this previous diagram where all of the browsers are hitting this R process at the same time, like, please give me a plot, please give me a plot, please give me a plot, we're going to handle it a little differently now. We're going to have R take the data set that's to be plotted, and we're going to have R kick it out to every connecting browser. And then from there, the browser will become responsible for regenerating the plots as you update those dropdowns. So it's all happening within those individual browsers.
DuckDB-WASM and Observable Plot
So that's the idea, cool, but how do we pull this off? So much like we use R packages to accomplish different needs that we have, there's some JavaScript libraries that we need to get the job done. And so we're going to need, what are we going to need? We're going to need a data manipulation library, something to select columns, filter data, aggregate if needed, and then we're going to need a data visualization library as well.
So for the data manipulation library, I chose DuckDB Wasm. So you might be thinking, like, DuckDB Wasm, you know, like, what does all that mean? Well, DuckDB, if you haven't heard already, it's a database, but it's not just any database. I think of DuckDB as, like, SQLite, but for analytics workloads. It's designed from the ground up to munch through the sort of queries that our shiny apps love to send.
But then what's the Wasm part? Wasm is a shorthand for something called WebAssembly, which you can roughly think of as, you know, assembly but tailor-made for the browser. And if those are still kind of, like, opaque descriptors, the important thing about, to take away from this slide, is that the synthesis of these two technologies, you take DuckDB and WebAssembly and you put that in the browser, you get stupid fast queries. You're not going to lose any speed in manipulating data and whatnot if we move to the browser, which is good.
For data visualization, I chose, because there's a lot of different JavaScript libraries for visualizing data, so I chose something called Observable Plot. It's a project that's built on top of a more mature project called D3.js, which is known for, you know, being able to create any sort of bespoke data visualization you could think of. Now, I ultimately chose this one as opposed to something like D3 for two reasons. One is that, look, I'm not really a JavaScript developer. Like, I'm an R developer primarily, and so I'd like for my JavaScript library to have as, you know, like, I don't want it to have a very steep learning curve, right?
So the nice thing about Observable Plot is that it's very similar in spirit to ggplot2. If we look at the code samples here, on the left we have, you know, some garden variety ggplot2 code in R, and on the right is the equivalent code using the Observable Plot API. And you can see that there's a lot of good similarities here, right? The whole idea being that in both cases, we have this plot that we're going to be composing by mapping different, you know, attributes of our data set to different geomes that come together to build this final, you know, composition that's the plot that we're after.
So the other thing is that it is flexible and expressive. We all love ggplot2 here, right? We love how it doesn't limit us in its ability to create visualizations and whatnot. And I didn't want to lose that when I moved from R into the browser. What's fortunate is that Observable Plot has a very expressive API that makes it easy to develop all the sorts of plots that we're used to making with ggplots. We can have our scatter plots. We can have our column charts. We can have our difference plots that are annotated every which way. And just recently, tool tips have made its way into the project. So that adds another layer of interactivity that you can inject into your visualizations.
Results of the refactored app
So we've refactored our app now, okay? So now we're using DuckDB Wasm and Observable Plot under the hood, and this is what the result is. The main difference is that we hit this send button, and what that's doing is sending the data set that's in R over into the browser. And then from there, these hex bin plots that you're seeing that get generated, this is all happening inside the browser now. R isn't doing that work.
So we've made these changes. Did it do anything, right? Well, this is a story that has a happy ending, fortunately, so we did improve on things. And we're going to see that in a new session duration plot here.
So this is the one that we started with, with the first app, okay? And we noticed that we have our reference duration of 93 seconds, and a lot of users are finishing well past that. We don't like that. That's not good. Well, here's where we're at now. You see this red line on the far side? That's that same reference duration, so that's 93 seconds, okay?
But then it's kind of confusing, because what's all these, like, little tiny bars on the left-hand side? Well, the important thing is that those are basically the operations that are needed to do to kick the data out into each of the individual browsers. From there on in, everything's happening inside the browser. So that line, that backlog of requests that forms when you just have, like, a typical Shiny app out of the box, that line's gone, because everyone's first in line inside of their own browser.
So this has a number of nice properties for us, right? We've achieved upwards of a 40% speed improvement. If you consider that in the worst case, users were finishing close to about 150 seconds before, we've brought that back down to 93 seconds, which is good. Very good. We've also achieved consistency, because now that all of this is happening in the browser, everyone's going to be finishing their workflow in around the same time. And again, we're talking about going from one to a hundred users, and now everybody's finishing in around the same time, with relatively, you know, small changes to the code base.
We've achieved upwards of a 40% speed improvement. If you consider that in the worst case, users were finishing close to about 150 seconds before, we've brought that back down to 93 seconds, which is good.
Benefits and caveats
So we've already talked a little bit about why this is awesome, but to kind of, like, put a bow on it, right? One thing that we've done is we've found a way to clear the prototype purgatory hurdle, right? And we all know what this is as Shiny developers. You build an app, it's great, people like using it, but then there's this does it scale question and stuff like that that we all have to deal with. Well, this is a nice solution to that.
In the case here, we were able to successfully scale to a hundred users. But there's another thing that I haven't really touched on yet, which is reduced infrastructure costs. And so what do I mean by that? So a lot of the times when we want to scale Shiny applications, what happens is that we usually resort to horizontal scaling, is what you call it, right? Which basically you have your app, and then you make, you know, however many copies of it, and then you load balance across each instance of that application. And that is a good strategy, it works, but the consequence is that it usually involves increased costs on your end to spin up all that extra compute. We don't have to do that here, because we're making strategic use of all the connecting browsers that are visiting our application, which is sick.
So this is all awesome. All well and good. But there's caveats, right? So there's always going to be caveats. First is that if you're going to use a strategy like this, browsers are not servers. Don't use them like that. If you start hucking petabytes of data at people's browsers, you're going to crash them and they're going to be mad at you. So maybe don't do that.
Another thing is that security still matters, especially if you're in the healthcare space or anything else that has, you know, sensitive data and things like that. You don't want to necessarily just be throwing data and exposing it out in the browser where bad people can do bad things with it.
And then another is that right now, so, like, this does have some minor hackery that I did to get this done. I have a link to the GitHub repo where this is where you can find this guy at. It's mostly just, like, minor changes to the HTTP UV package, just, like, adds a MIME type for it to accept WASM files and stuff like that. But it's still kind of annoying to set up up front.
But those are really much the only caveats I have. Go ahead and try it. It's fun. And then let me know what you think. Those are my contact details, if you'd like to talk to me more about this kind of stuff or other data-related things. And then there's the link to the GitHub repo as well. But that's all I have for you guys. Thanks again for joining me.
Q&A
We have time for a couple of questions for Joe. The first one is kind of a cheeky question. Once you're using JS libraries, DuckDB, and WASM, what value is Shiny still providing as a web framework over something like React?
So I love that question, right? So I think that what a strategy like this shows is that with all of these technologies, it's important to start thinking about the strategy behind your application. A lot of the ideas that I brought up, like, browsers aren't servers and stuff like that. That's part of the thing that I would respond to with why I still think that some things should still be happening on the R side. Certain data manipulation and things like that may need to stay on-prem or in your own servers and stuff like that, not happening in the browser. But I think that it's more like, we're not looking to replace one thing outright with the other. It's a good strategic tool to have in your belt.
Your next question. Did you measure how long does it take to render on the client? And can you share how large your data set was?
Yeah. I'd have to, so as far as how long it took to render the plots, since it's all happening inside the browser, if we, and I could just jump back to the slide real fast, so how long it took to see the refactored app generate the plots, that's how fast it's going to be. Because it's all happening in individual browsers. There's no more competing for a single process. As far as how big the data set was, here it was about 56k rows, I believe.
And your last question is, are there caveats to keep in mind when working with different browsers? Almost certainly. But I'm not smart enough to speak to that. I'm sorry.