Introducing Bookmarks | RStudio Webinar - 2016

So, I think many of us have experienced this where we've created a cool shiny application and you want to share it with somebody. So here's an example, this is from our shiny gallery, somebody who created a data visualization application for data from the Center for Disease Control.

And let's say you've created a cool app like this and you want to share it with somebody. Well, doing that right now is very easy, that's always been one of the really easy things to do with shiny. You just send somebody a URL, they go to it and they can see your app.

But if you wanted to share not just the app, but the work that you've done in the app, like let's say I've interacted with this application and I've selected a particular disease, date range, location, and so on, and I want to share that with somebody, the particular state of this application, that isn't as easy to do.

So you might find yourself writing emails like this, and I'll just read this out loud. So this is to the boss, subject pertussis data, hi boss, here's the app that shows the latest data. To see the results, go to this URL, for disease name, select pertussis, for choose date range, make it start from 2014, for plot type, choose weekly data by year, for plot options, check the date, include alert thresholds, and then you should see what I'm seeing, enjoy your holidays.

So you may have, if you've written shiny applications to share with people, you may have done this before, and, sorry, this control panel keeps popping up, so you may have done this before, but it would be nice if you could write an email that's more like this, here's the latest data, just send a URL, enjoy your holidays.

And that particular application, the example that I showed is just, there's only four settings on there, but there's still, you have to say, do this, and this, and this, and this. And for more complex apps, you might have an even longer set of instructions for how to get to a particular state.

And for more complex apps, you might have an even longer set of instructions for how to get to a particular state.

Introducing bookmarkable state

So in order to do that, to send somebody not just an app, but also the state of the application, we've introduced this feature called bookmarkable state, and this was released with shiny 0.14, which came out in September, I believe.

So now I'm going to illustrate this with a very simple example, I'm going to illustrate how you use it. So I'm going to start with a very simple demo app that should be easy to follow.

So this is an application with just one input and one output, and I'll show you it running live here. Okay, so we'll run that, and I enter text, hello world, and it outputs the same text down here. So that's the basic app.

Now if I want to be able to bookmark the state of this app, there's just a few changes that I have to make to the code. So this is what it looks like, and this is with a, this is for a single file app, just an app.r.

So here's the few changes that I need to make. First off, the user interface, the UI part of it needs to be a function that returns the UI components. So instead of UI being fluid page, it's a function that returns this fluid page, etc., etc. It takes one argument called request, which, that argument doesn't actually get used, but the function needs to take that. It doesn't get used in this case, but there's other reasons to use it that aren't related to bookmarking.

The next thing that we need to add is a bookmark button somewhere so that the user has a way to actually tell us that they want to bookmark a particular state. And then when you call shiny app, you just need to add this part at the end, enable bookmarking equals URL. You just need to add that one argument.

And so this is the single file app, the app.r version. If you want to do a, if you're writing an app with UI.r and server.r, it's very similar. You say UI is a function, add a bookmark button, and then you have, in your global.r, you say enable bookmarking URL. This time it's a function called enable bookmarking instead of an argument to a function. So that's all you need to do.

I'll run my code here. All right, so hello world. And when I click on bookmark, this dialog box comes down and it gives me this bookmark application link. So I can copy this and I'll paste it into a browser. And the application is restored with the same state that we had over here. They both say hello world.

And so for implementing bookmarking for simple applications, that's really all you need to do.

URL-encoded vs. server-saved bookmarks

Okay, oh yes, and I'll show you how this works in an application that's deployed in our gallery. This is an example application. Again, this is also very simple, but I want to show you what the URL looks like. So there's just a slider here that controls the number of observations. So I'll just move it up to, let's say, there's 241. So and then I hit bookmark. This is what comes down. This is this URL. And I paste it into a new browser window. I get the same result here.

All right, so this is an example of a URL encoded bookmark. And what that means is that, what that looks like is, first you have your, you know, this is your path to your application here. And then after that, there's this question mark and a bunch of stuff after it. So what this is saying is, yeah, I won't go into the, you know, you don't have to generally know the details, but I'll explain this to you briefly, is this means these are the inputs and then there's an input named N with the value 1, 2, 3, and input named text with the value ABC. Okay, so that's what that would be for this particular example.

But that's not the only way that we can do bookmarking. So another way to do it is to use bookmarks that are saved on the server. So for these type of bookmarks, instead of encoding all this information about these inputs in the URL, all of that is saved on disk and then it's given an ID. And so, when you want to send a URL to somebody with that state, the URL is, just includes this part that says state ID equals and then, you know, whatever that unique ID is.

Okay, so, oops, all right, so I'll talk about some of the, in depth about some of the differences between these two types of bookmarking in just a bit. First I'm going to show you how to save to server bookmarkings or how you use it. So it's very similar to the URL-encoded bookmarking. So the UI just takes, you know, this function, add a bookmark button, the only real difference is you need to have this argument enable bookmarking, set it to server instead of URL. And that's it, that's the only difference. You know, from the perspective of somebody who's writing the code.

All right, so here's some of the differences between these two types of bookmarking. For the URL-encoded bookmarks, there's, as we said, there's no state saved on the server, all of the state is contained in the URL and if it's saved to server, it's obviously the state is served on the server. For URL-encoded bookmarks, the URLs can become long. So if you have a lot of, you know, like those examples that I showed you just have one input or two inputs, but if you have a lot of inputs for your application, those, or, you know, if those inputs have very large values, like, you know, if you've got a text input with a very long string that somebody's typed in, the URL can become very long and that can be a little unwieldy if you're sending it to somebody. And if it's saved to server, it's always a short URL, the state ID is always the same size.

And because the URLs are long with URL-encoded bookmarks, you can run into problems with some browsers. Some browsers can only have, take URLs that are up to about 2,000 characters in length and if your URL exceeds that, I believe it'll just get truncated and so when you try to restore your application, it probably won't restore correctly. And if the state is saved on the server, there's no limit to how much data you can store. For URL-encoded bookmarks, you can't save files that have been uploaded in that URL, but if it's saved to the server, you can.

And finally, these URL-encoded bookmarks, as of now, they can, well, these can work on any hosting platform. So like ShinyServer or ShinyApps.io or RStudio Connect, whereas the saved server bookmarks work on ShinyServer only right now.

Deploying bookmarkable apps

Okay, so I'll go into that for a little bit more depth here. When we talk about deploying the bookmarkable apps, the various hosting platforms, here's a table for what you can do on the various hosting platforms. So when you're just running these applications locally on your own machine, like I'm doing right now in these examples or when you're developing an application, both types of bookmarking will work. If you have ShinyServer Pro, you can use both types of bookmarking and same is true for ShinyServer Open Source.

If you're deploying your app to ShinyApps.io, you can use the URL-encoded bookmarks, but the saved server ones are still in development. And the same is true for if you're using RStudio Connect. The URL-encoded bookmarks will just work, but the saved server bookmarks, there's still some work that needs to be done for that to function.

Advanced bookmarking

Okay, so those are some very simple apps, applications, and for a certain class of applications, it is that easy to turn on bookmarking, you know, just to add a few lines of code. But for more sophisticated applications that have more complex behavior, there might be some other things that you need to do. So that's what I'm calling advanced bookmarking.

So now we're going to get into some technical details about these applications. So the simple application that I showed you before is what I'm calling a fully input-dependent application. This term is just something I made up, but there may be a better name for it, but I couldn't think of one offhand. So if you've got suggestions, let me know.

Alright, so for this type of application, the state of the outputs at time T is fully determined by the state of the inputs at time T. So that means that, you know, whatever the value of the input was before, if I had typed in something besides hello world before, and then I change it to say hello world, whatever the previous value was, that does not impact the current state of that text output. Alright, for these types of applications, bookmarking should just work.

And here's some diagrams. If you've read the article that we have in our Shiny Dev Center about reactivity, these diagrams may look familiar, but this is a very simple relationship between an input and an output. So we've got input $n and output $text. So it's just render text takes input n and that's the value of output $text. So whenever input $n changes, output $text changes. And the value of this doesn't depend, it depends only on the current state of input $n.

And you may even have a chain of these things. So here is input $n is being used by this reactive called n1, and then output $text uses n1 to get the content that it's going to display. So this is sort of a chain with this reactive value, input $n, reactive expression, and then this output. But again, this has a similar relationship where output $text depends only on the current state of this input, and it could depend on the state of multiple inputs as long as it's just the current state of them and not any previous states.

Okay, now contrast that with what I'm calling a partly input-dependent application. Again, if you've got suggestions for better terminology, I'd be happy to hear that. So the state of the outputs at time t is only partly determined by the state of the inputs at time t. So other things may influence it, including inputs at previous times or perhaps, you know, external data sources. And for these applications, they'll require a bit more code for bookmarking to work.

All right, here's an example. I won't run through the, I won't walk through the code, but I'll just run it and it'll be clear what's going on here. This is what I'm calling an accumulator, it's just adding values. So I've got the slider here, make this bigger, oops, okay, that didn't zoom this thing.

Anyway, so it's set to 50 right now, and if I click add, it adds it to the sum. And if I set it to, you know, 100.

Winston, excuse me, I don't know, I'm not seeing your screen. Oh, I'm sorry. Did it disappear? Did it just disappear now? Yeah, it just disappeared now. Okay, okay, I don't, yeah, sorry, the presenter controls are going a little bit wonky. Okay, let me, I'll try running this again.

So, I've got a sum right here, and I've got a current value, and if I click add, it'll add that current value to the sum. So I've added 50 to it, I can say add 80 to it, and then I can, you know, let's say add 20, and so on.

So this, in this application, the state of the sum depends on the value of this input, not just now, but also at previous times, at previous times when I clicked add.

And if I bookmark it, I'll copy this link, I'll paste it into my browser window. And as you can see, the sum that it's displaying is 50, which also happens to be the current value of the slider. So that's, this is, in this case, bookmarking did not fully save and restore the state of the application. All right, so let's talk about why that is.

All right, so this is the server part of the, this is the server part code for that application. It's not too complicated. These are the important parts. So first, there's this reactive values that stores sum. I've got an observe event, which every time the add button is clicked, it adds the current value of input n to the sum. And then, and then when val set sum changes, that causes this render text to be triggered and it updates the value that's printed on there, the sum value.

Okay, and this is what we saw before. The sum, let's say, was 200. I clicked, and the current slider is on 5. I clicked bookmark, and then when I restored it, I ended up with a value of 50 here, which is not, that's not correct.

Okay, so in order to deal with this sort of problem, there's some callbacks that, there's some functions that you can use to register callbacks when an application is bookmarked and when it's restored. So, because in, for this particular application, we need to save some extra state. That's not just the current input values. The current input values are saved automatically when you bookmark. But this, we need to save some other information as well.

So, these two callback registration functions are onBookmark and onRestore. So, what this one does is, we call onBookmark and we pass it a function that takes one argument state, and what it does is, when it executes this, when the user clicks the bookmark button, so, and it saves val$sum into these state values. So, this state$values is a, it's essentially a list, and you can just put in arbitrary information in there. So, state$values, $currentsum, we're just saving the current sum in there. And, when we restore the application, instead of starting up here where the sum is 0, with starting with that value, what we end up doing is, we extract the saved current sum out of the state and put that in val$sum.

Okay, and this is, well, this is part of the, this will get us partway there. Now, let me show you what happens if we run this.

Okay, this is the application list. It's the same code, except there's these bookmark and restore callbacks. Okay, so, let's say I click add a bunch of times, bookmark it, and if you can see this, there's this extra part of the URL called values, and $currentsum is 200. So, that's the value that I saved in there with that on bookmark callback function. If I paste that into my browser, here's what I get. It looks a little different. So, here the sum is 200, and here the sum is 250. So, that's possibly a little bit better, but that's not quite right.

And let's see why that is. Okay, well, what's happening is that there's the, after the application is restored, input $add has some value, like 4. That's the number of, actually, that's the number of times, it's an action button, and that's the number of times the action button has been clicked. And because it has a value of 4, it triggers this observer. It triggers this observer. So, the sum before this observer fired was 200, and then, that's right after the application was restored, and then this observer gets triggered, and then the sum goes up to 250, because it thinks that we clicked the add button one extra time.

So, and this is because of the way that action buttons work. They have some numeric value that represents the number of times they're clicked, and for this observer event, if that number, if that value is above 0, then it will execute this expression here. So, one way out of this, it's not the only way out of this, but one way out of this is to not bookmark the add button. So that when the app is restored, it will start with a value of 0 and won't fire this observer event.

In order to exclude a value from being bookmarked, there's a function called setBookmarkExclude. So, we're just saying setBookmarkExclude, and the name of the thing that we want to exclude is add. And this can be, you know, you can pass in a vector of names. We're just excluding one value here.

So, let's try running this. Okay, so, sum of 200, hit bookmark. Copy this. And in the new browser tab, the sum now displays 200. And if you were watching very closely, this is the current URL that I have. Inputs n equals 50, and then there's this values current sum equals 200. And in my previous attempt, there's also this add equals 4 that was being saved in the state there. In my current attempt, that's no longer there, and that's because we excluded them. So, that worked.

Alright, and this is again what we saw. We started with this state of the application where the sum was 200, we bookmarked it, and then when we restored it, it restored the correct value of 200.

Callback functions for advanced bookmarking

Okay, so, so for these sorts of applications, where the state of the application is a little bit more complex than those fully input-dependent ones that I described in the beginning, there are these useful callback functions. So, there's onbookmark and onbookmarked. This function, so the way that these pairs of functions work are when somebody clicks a bookmark button, just before Shiny actually saves the state of the application, it runs this onbookmark callback, and then afterwards, it runs this onbookmarked callback. So, there's some reason that you'd want to use these two different functions. And for restoring, there's a similar pair. There's onrestore, which happens just before the application is restored, and then onrestored, and that gets called just after the application is restored.

And in the documentation, there's some examples that explain when you'd want to use these two different things. And finally, if you want to exclude values from bookmarking, there's the setbookmarkexclude function. So, I excluded the action button, that add action button, but there's, you know, there may be other types of things that you'd like to exclude.

Okay, so, you know, going back to our original story here, and you've set it to a particular state. So, now that you've learned something about bookmarking, you probably could write an email like this. Here's the latest data, and just send a URL, and that will send not just the application, but the work that you've done in the application. And if you'd like to learn more about it, you can visit the Shiny Dev Center. We have an article section, the articles page, and then there's a section in there called bookmarking state. And that includes some more technical details if you'd like to learn more about it. So, there it is. Thank you very much.

Q&A

Okay, so here's one of the questions is can this kind of bookmarking be done with Shiny and R Markdown? And I'm pretty sure the answer is yes, it's been tested before. There's I'm trying to remember in order for it to work properly you need to set an option in R Markdown to have it be pre-rendered. I forget the details about that though and hopefully we can look that up and give some more details about that. That was actually a new feature that was implemented in the last couple of months for R Markdown I believe.

Can I bookmark the contents of an uploaded file if I were to assign the contents to a variable? Yes, you can do that. So, I think this question relates to what I said about a URL encoded state not being able to contain a file. So, yes, if you're able to take out the contents of that file and save it save it to a separate value, then you could that could be part of the bookmark. But you'd have to use that onBookmark and onRestore because the file with file uploads the file, its content itself is actually not an input value, it behaves a little bit differently and so that part doesn't automatically get bookmarked when you say enable bookmarking. What you need to do is use the onBookmark callback function extract that file content and then save it in state$values$ file content and then with the onRestore then you'd restore it there.

Can bookmarking be done with Flex Dashboard? I'm not 100% sure of this. I believe it can because Flex Dashboard is using R Markdown I believe it also requires this pre-rendered this shiny pre-rendered option to be set.

Okay. In the save to server mode, is the user able to download the file that contains the saved state of the application in addition to getting the app URL? I believe this is technically possible because the state does include the directory where all these files are stored. Whenever a state is saved to the server a new directory is created and all the necessary information is saved in there and it should be possible to retrieve that. There's not a simple API exposed to do that. It is possible to find that directory you can zip up all the files in that directory if you wanted. Yes, it's possible, but it's not trivial for most users I think.

Are there performance implications for using bookmarkable state? For bookmarking, there's no if you just turn it on for your application, there is no overhead performance penalty. It's only when the bookmarking actually occurs that there's any computation. In many cases, if you're just saving the values of inputs, it's a very small amount of data so it happens very quickly. The one place it can get large is if you have large uploaded files. If you save the state it will make a copy of these uploaded files. That could potentially consume a lot of disk space on the server.

Is there security value of using the server versus URL method? With the URL encoded bookmarks all of the state is at least the state for all the inputs is encoded right there in the URL. If somebody else is able to get a hold of that URL they can see what inputs one has set. Depending on your situation, it may or may not be a security issue. If the app also requires a login to Shiny Server Pro or RStudio Connect then that's an independent thing. If the person is not able to access the application then it's possible that information in the URL is not going to be all that useful for them. For server side bookmarking I don't think there should be any security issues other than the state obviously is saved on the server so if somebody has access to the server they might need super user access then they can look at different state files but if they have super user access to the server you have security issues anyway.

If you have dynamic UI elements i.e. one UI element reacting to another, is there any issue with the order in which the UI elements are restored or are they all restored at the same time? Restoring dynamic UI was actually one of the challenging bits in implementing bookmarking for Shiny to make sure that that behaves seamlessly. Typically it depends on the specifics of your application. We try to make it so that if your application has any dynamic UI parts it should just work. Even if some of them depend on some other dynamic UI inputs that should just work in most cases. I believe we talked about that in some of the articles on the Shiny Dev Center.

Have you experimented with auto bookmarking where every time the user does an action the state is bookmarked and you can play back a session by auto playing the saved state URL? We actually have not tried that. In the documentation there's an example where when the user interacts with the application instead of having them click on a button for bookmarking it actually changes the URL in the browser's location bar. That can be updated automatically. In fact, I can probably just find that. This is an example where when I type in something like bcdefg the URL updates as I'm typing. Hi everyone. I can click this checkbox and as you can see the URL updates.

This doesn't exactly answer the question that the person asked but something similar can be used where it reacts to every time somebody makes a change to inputs. But you'd have to figure out your own way of saving the information.

How does the server admin manage all saved states? Right now it's each of the saved states is just a directory with that unique ID. For each application there's a directory that contains subdirectories with those names. Right now we don't have for ShinyServer and ShinyServer Pro we don't have a really easy administration tool for that. Hopefully we will at some point in the future. Unfortunately what I can say right now is that you could sort them by their dates and do something with it that way. Like if you want to just call bookmarks that were created a long time ago or were only accessed within or were not accessed recently you should be able to do that. But there isn't a nice front end tool for that. But that's something that hopefully we'll be able to bring to the various hosting platforms.

What about timeout cleanup of saved states? I think this is a similar question. Asking a similar question. The answer is you'd have to use it the best way right now is to use standard Unix administration tools.

I see one more here. When bookmarking using state IDs can you examine input parameter values server side in some way? If this question is asking what I think it's asking I think when the onbookmark callback is invoked or when the onstore callback is invoked you can look at the state and if this question is asking about examining those values offline outside of a session the answer is yes it is possible when the files are saved on the server that state object is saved as an RDS object which R can just read in read RDS and then you can examine the contents of it. It is possible to look at the contents of those input values.