Barret Schloerke || Maximize computing resources using future_promise() || RStudio
00:00 Introduction 01:45 Setting up a multisession using the future package 02:05 Simulation using two workers 04:14 Simulation using 10 workers 05:20 What happens when we run out of workers? 05:35 How Shiny handles future processes like promises 07:16 Introduction to future_promise() 07:45 Demo of the promises package 09:21 Setting the number of workers 10:40 Demo of processing without future_promise() 14:11 Wrapping a slow calculation in a future() 14:53 Demo of processing using Plumber 16:25 Considerations on the number of cores to use 17:21 What happens if we run out of workers? 19:44 Decrease in execution times using future_promise() In an ideal situation, the number of available future workers (future::nbrOfFreeWorkers()) is always more than the number of future::future() jobs. However, if a future job is attempted when the number of free workers is 0, then future will block the current R session until one becomes available. The advantage of using future_promise() over future::future() is that even if there aren’t future workers available, the future is scheduled to be done when workers become available via promises. In other words, future_promise() ensures the main R thread isn’t blocked when a future job is requested and can’t immediately perform the work (i.e., the number of jobs exceeds the number of workers). You can read more about the promises package here: https://rstudio.github.io/promises/articles/shiny.html And you can learn more about Shiny here: https://shiny.rstudio.com/ Got questions? The RStudio Community site is a great place to get assistance: https://community.rstudio.com/ Content: Barret Schloerke (@schloerke) Design and editing: Jesse Mostipak (@kierisi)
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Okay, so future is a wonderful R package that does like a lot of voodoo behind the scenes so that you can easily run code in a separate R process. And I think that in itself deserves a ton of praise because the API is so simple and is so useful. I actually don't even use the shorthand API because I like the very direct explicit future where you say, hey, run this code in the future, and then go get a value. And I think that in itself is a wonderful handshake and I really like this and I really enjoy embracing this.
This is very useful if you have like, hey, I run a, I want to run 100 simulations and I have 10 cores. Let's, you know, make a plan to use all 10 cores and just churn over those and we can actually use R on all separate 10 cores or in 10 different R processes. Hopefully your machine is smart enough to use all the cores. But like typically R is single, single worker is the way I want to think of it. And so if it's a single worker, you know, you churn through it serially. But if we were able to use future, we can now churn through it in parallel using as many workers as we have available. And I think that's awesome.
Setting up a multisession using the future package
It works out really well. And we can actually kind of take a quick peek at this. So we library future and then we can set up plan and say multi-session. It's just a good default. And then we can say workers is two for the demo purposes, or for the other example, I can set it to be 10. You know, that works great as well, or a hundred, you know, it doesn't matter, but two works. And then here I have my like simulations from one to 10 that I want to run. And I'm going to create the future. I'm going to set up the future. I'm going to say it was created, and then I'm going to return that future object. And then finally at the end, we will look at all of the values for each of the vows.
So let's library plan run vows. And it immediately goes into the first two are created right away. And then we start to go through this just a little bit slower, but in pairs of two. And that's really neat. Like if we're looking at it with assist sleep too, that should have taken what 20 seconds. But if we do a start is sys.time and end is sys.time. What is it? End minus start. Maybe this will work. Try it again.
So it took 10 seconds and this is awesome. We have two workers and well, it took nine seconds, not really 10, but we have two workers and it all finished up very quickly in parallel, not 20 seconds. That's the important part. We can get in later to like where that extra second went, but I'm not too worried. It's roughly 10. So that's the good part. And if we get the values, we have one to 10. Perfect.
This is really neat. But one of the things that was interesting is when you notice and you watched how it was working, I could not actually execute this end or this end minus start until the LApply had finished processing. That's a little weird because I'm saying, Hey, dear future, please run this in the future. And this is because it actually had ran out of workers to submit jobs to. So if I switch this and I say workers is 100 plan, I start do the vows and should go much, much faster. Oh, let's actually cancel that. Let's say workers is 10, because it's actually making 100 R sessions. We don't want to do that. Um, so let's set it up to be 10. Now they're created very quickly. And so let's actually just redo the whole thing because I didn't even hit the end time point. So it took two seconds, you know, submitting the jobs is not trivial, takes time. But it's a lot faster than if we are to do this serial. So that's good. And we can look at the values. Everything is great.
What happens when we run out of workers?
So that's a lot of fun. Like it's, it's really neat, like how all of that is able to be done. But there's that funny situation of when you run out of workers, and what happens, you know, when we run out of workers, and my thought is it just immediately returned. And turns out, it actually blocks. Blocking has very big implications for Shiny and for plumber.
Shiny will actually treat future processes very similar to a promise. And a promise is a thing in R that says like you can chain these promises, but you can also kind of interleave the processing of promises, promises kind of jump into later. But the idea is that you can, you can interleave the processing of promises. So if you have a job that takes a very long amount of time, you could actually break it up into a chain of multiple promises. And that would allow other things to work in between, rather than waiting for your job to finish completely.
This idea can also be scaled up for Shiny in that like, what if we have multiple users coming in? If this user comes in and says, please process my model, and it's one big block of code, no one else gets to do anything, all of your cores are being underutilized, nothing. And so instead, you could use promises to say, if you could break up that section of your model or processing of that model into multiple, multiple promises, then other users can come in and just say, oh, give me hello world, give me hello world, and no problem, they don't have to wait till the end.
So I think it's really cool. Future, we treat just like a promise, and it gets upgraded. And the result is, is handled accordingly. The bad part is future blocks when submitting a job if there's no workers available. So there is a brand new function in the promises package called future promise. And a future promise for all practical purposes is just, I promise to execute this in the future.
And a future promise for all practical purposes is just, I promise to execute this in the future.
Introduction to future_promise()
So for the majority of what you'll do, whenever you would normally say future parentheses, you can kind of replace it with future promise parentheses. So in our code, if we were to come in here, and let's actually copy this and give me a script and library promises, clear out output. All right. So we'd say library, future library promises, set up our multi-session, let's actually adjust our workers back down to two.
Okay, and then we will adjust our future to be a future promise. Let's try running this. So the time finished right away. And actually, I was able to get control back to the console right away. And then these messages started just showing up when it started completing and submitting these jobs. And I think that's a very subtle difference, but it's a very important difference. Because if I run this again, of calculating the vows, I can actually come in here and say one plus one, say two plus two, two plus three. And I'm able to submit this work while waiting for the future to finish. And now my main R session is not blocked. This is super cool. This is that, wow, because now I can process other people's work while not waiting for that big future job to finish or to be submitted. And that is very, very beneficial.
Setting the number of workers
So workers is an arbitrary thing. You can set it to what you want. It's dependent on what you want to use them for. So if you have things like submitting data to MySQL, or not labor intensive tasks, but you need to do a bunch of them, then maybe you have like two to three of the amount of cores that you have. Because they're not going to be utilizing that much work and it's okay. You could even go higher than that if it's not a real intense task. But if you're counting all the numbers in Fibonacci, it's very computation expensive, then maybe you do cores minus one. So if I have eight cores on my machine, I might say workers equals seven so that I can click on my computer still. Otherwise you'll have all of your cores maxed out and your computer's rendered useless. But it's up to you as to how you want to use it. Workers are not cores directly, but your computer will try to offload them and have them process because it's not in the computer's best interest to put them on the same core.
Demo of processing without future_promise()
So plumber. Plumber runs on HTTPUB, just like Shiny. So a lot of these concepts work just the same as if you're doing plumber API or a Shiny application. The concept of working and using futures and promises just extends from one to the other. So in plumber, let's imagine that I have two functions that I want to expose in an API. Plumber is a R package that will allow you to create a web API from R code just using a couple little decorators. In this case, I have two functions, the fast calc and slow calc, and I can expose them as routes with the at get fast and at get slow. And for me, in the case of the demo for this, the ID is just representing the submission order, but calculation can do whatever it wants with the parameters, things like that. But then that way, when you see something like slow one and slow two, we know that slow two was submitted after slow one, even if they are very close together in submission time.
Cool. So plumber handles this, and in this case, we're going to have four routes. We're going to have a fast route, a slow, a slow, and then a fast route. And if we were to process it, all four submissions kind of done at the same time, it would look like this because R has only one worker. So we do the fast route, and then it responds, and then okay, it's done. Then we do a slow route, and then it processes, and it takes a very long time, and no one else can do anything, and slow three is just sitting there waiting to do work. Then it's done, and then we respond. Great. Slow three comes in, and it starts to process, and it is very slow. It's taking its time, and fast four has to wait for all of this processing time to work. And then okay, we're done. And then now we can respond, and then fast four can come in, and it processes really quickly, and then it can respond, and we're done.
So great. Awesome. Those four requests came in, and they took about 20 seconds. If we think that slow takes 10 seconds to calculate, and fast is roughly instantaneous. So the fast comes in, it processes right away. Slow comes in, takes 10 seconds to process, and then slow three had to wait that whole time, and then slow four had to wait even all 20 seconds for it to be processed, even though they kind of arrived at the same time.
This red area is bad. This is something that we can optimize. I can't necessarily make slow calculation faster, but I can reduce the waiting time, and that's a goal.
Wrapping a slow calculation in a future()
So with plumber, we can do this using future, and this is just talking about using futures, and so like how they can work with plumber and with Shiny. A lot of people don't know this, so like if you're compiling a PDF in Shiny, if you return a future from your reactive, or your output, or an observe, it will put it in this promise, and it will listen to it. It's awesome. I read the docs, but you can offload things quite well. The only change in this situation is that we're going to wrap our slow calculation in a future, and then we will resubmit it, and from before it took 20 seconds, and now it will take 10 seconds, and there's been fast4 got reduced by 20 seconds total, and slow3 did not have to wait, and so it was 10 seconds faster. So all the wait time in this example was able to be removed. We can't reduce the processing time, but we can at least like remove or get rid of the wait time. That's awesome, and so this is how future can help you do that.
Demo of processing using plumber
Cool. So how would this look like with plumber? Plumber is typically, or is always in the main R session, or the main worker, and we can add extra workers. FutureWorker1, FutureWorker2. So having those same routes come in where the slow is actually wrapped in a future, it will look like so. Oh yeah, the FutureWorker, we're only doing it in the process step. So okay, fast comes in, it is processed really quickly, great, we respond, awesome. Slow comes in, hey it's a future, let's offload it, great. This allows the main R process to come in and do the same thing with slow3. Since we have enough workers for the jobs, then fast4 is able to process immediately. slow2 just happened to finish, so we can send it off, and same with slow3. Like a lot going on, a lot of moving parts, but it's really nice because it allows to offload that slow processing somewhere else to free up that main R worker. So you can respond to health checks, you can, you know, have other users in Shiny do simple reactivity, and it's not going to be a big deal. You're not waiting for someone else to finish running their model. Why should someone else's Shiny session alter my Shiny session? And futures and promises will help you do that.
Considerations on the number of cores to use
So that's where we get that 20 second save, by being in parallel. So some limitations you were talking about, how many cores do I do? Well, you know, we saw I was trying to add 100 cores at one point, and my computer just kind of blew up a little bit because it was spawning 100 R sessions. Like, maybe not the best idea. Two to three is pretty good, or one to one if you're computationally intense for your processing. Spawning a future is not instant, kind of like a quarter of a second, but if a quarter of a second really doesn't mean much to you for time, then future is perfect. If a quarter of a second does mean a lot to you, maybe don't offload it to an external R session, because maybe that communication overhead is too much, and you're already losing time. So that's kind of where it leads into the processing power is finite, and memory size is finite. Maybe if your models take up 10 gigs of memory, maybe you only use two workers. It's still better than zero.
What happens if we run out of workers?
What happens when I run out of workers? So let's pretend we have six slow routes all being handled by future, and what would happen? By slow three, we will run out of workers. So slow one comes in, offloaded worker one. Great. Slow two, offloaded worker two. Slow three, blocking the main R session. And the main R session is still blocked, because now slow four is blocking. Slow five is still blocking the main R session. Slow six is still blocking the main R session, because it can't be offloaded somewhere. Now it's been offloaded. Fast seven is still blocking the main R session, but it's kind of free. And then now we can undo the result of the future values, and now the main R session is unblocked. It's kind of free. Phew. Now we can process slow five, and now we can process slow six. But while we were waiting to offload slow three, four, five, and six to offload the new workers, the R process was not usable, the main R process. So in the case of plumber, you cannot do a health check. In the case of Shiny, I cannot load up a new R session, or a new tab. Like it's blocked. Nothing can be done.
And if we were to visualize this, you know, the processing time takes 30 seconds in total, offloaded in pairs. And there's actually a new thing where the results of slow one and slow two do not get returned until 20 seconds in. That was when the main R session became free. And slow three and four had to wait, slow five and six had to wait, fast seven had to wait, and that's not good. Like we want to try to reduce that wait time as much as possible. To add in as well, this is the area where there were more future requests than future workers. This is also the area where the R session plus the final end is where the R session is blocked, the main R session. That is where things are unusable, even if you're only just handing them off, or you're waiting. This gray area is bad, and we'd like to minimize it as much as possible.
Decreasing wait time with future_promise()
Cool. So this is where future promise came in. Promise that executes using a future. If we were to do this with future promise, the execution time would look like this. The items slow one, slow two respond immediately, slow three and four just like were before, but the big one is fast seven is now actually executing immediately. It is not waiting for anyone.
Fun animations and also more fun animations. The big difference here is where the gray bars are at. In this case, the gray bars are only at the start when we're resolving the values for one and two, and resolving the values for three and four, and also for five and six. Can't get rid of those. But in between, completely free. You can run health check routes on plumber. You can start new sessions in Shiny. You're not blocked. Everything is awesome. Anytime you say future parentheses, just say future promise. Solve a lot of issues, and for the most part, as long as everything's scoped just like it should be, then everything will behave as expected. For the most part, it's just a one-to-one drop-in.
Anytime you say future parentheses, just say future promise. Solve a lot of issues, and for the most part, as long as everything's scoped just like it should be, then everything will behave as expected. For the most part, it's just a one-to-one drop-in.
For comparison, let's see how the animation works for this one. So we have all of our one through six slow routes, and then a fast route. And the addition here is that we're going to have this promise queue of promise to compute with future. So it's a pre-queue, not a post-queue. So we have this pre-queue because there's no workers available, so we'll just say I will, I promise to do this. So we can now execute fast seven immediately, and then slow one, and slow two have finished processing after 10 seconds. And the order may be a little fuzzy here, but for the most part, they go in and out in pairs. You can only animate so well in Keynote.
But throughout the whole process, the main R session is free 98% of the time, not blocked 60%.
