Eric Nantz - A New Era for Shiny-based Clinical Submissions using WebAssembly
In life sciences, Shiny has enabled tremendous innovations to produce web interfaces as front-ends to sophisticated analyses, interactive visualizations, and clinical reporting. While industry sponsors have widely adopted Shiny, a relatively unexplored frontier has been the inclusion of a Shiny application inside a submission package to the FDA. The R Consortium R-Submissions Working Group has continued the momentum of previous submission pilots, such as the successful Shiny app submission to the FDA in 2023. In this talk, I will share the journey of how we used containers and WebAssembly for a new and innovative approach to sharing a Shiny application directly with the FDA, paving the way for new innovation in the clinical submission process. Talk by Eric Nantz Slides: https://github.com/rpodcast/shiny-webr-posit2024 GitHub Repo: https://github.com/RConsortium/submissions-pilot4-webR
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hello friends, yes my name is Eric Nantz and I am super excited to be telling you about a journey that we have started about looking at a new era about using the intersection of Shiny and Web Assembly for the potential of a clinical submission pipeline. So let's first do a little story time here and let's put yourself in a situation where there has been a pivotal study that has just entered its data lock, it's gotten all the results, everything is available, and it is time for review.
The results are in. This is one of the most exciting times in our industry to help bring an innovative medicine to patients worldwide and so it is time to start digging into those results. Is there a few results? Not always a few results. Oftentimes there are many results. If you are familiar with the concept of tables, figures, and listings, a pivotal trial could have hundreds, sometimes even over a thousand of these results and in the history of the pharma industry these were coming, not to dissimilar what you see here in paper format, and these were delivered to regulators even in truckfuls of them in an actual submission. I don't know about you but when I see this I would feel just a tad bit overwhelmed trying to figure out what are the insights from these results.
Now of course in technology, in the world of technology, we have the advent of digital methods and now technically speaking you could probably fit all those thousands of results and more even on a little USB thumb stick. So yes we are in the digital age but those are still static type representations or just happen to be in electronic formats like RTF or PDF and what have you. Is there a better way that we can make these results easier to digest, easier to review? My hypothesis here is yes there is because we have the Shiny package and more recently we now have WebR and the combination of these I think can bring us to this new era.
My hypothesis here is yes there is because we have the Shiny package and more recently we now have WebR and the combination of these I think can bring us to this new era.
The R Submissions Working Group
Now it's one thing to have this idea but how are we going to put this in action so that we all in our industry can benefit from looking at this together? That is where the efforts I'm talking about today have been part of what's called the R Submissions Working Group which is a fully sanctioned effort from the R Consortium and the key part here is the collaborative nature of this because this is not just sponsors or industry members it is also regulatory members that are hand in hand joining on this journey. It is also being able to create submission deliverables with open source in mind. This is not a proprietary solution we want everything to be transparent and again collaborative and we also want to emulate what it is actually like in the day-to-day of a health authority regulator reviewing these results that we are producing in this working group. We don't want to just do like a what if we want to really get down to the nuts and bolts of what it's like to look at these results and this is one thing I'm so excited about innovation is not just a buzzword here we are encouraged to push the envelope here in this environment which is not always the case when we get to our day-to-day of clinical work.
Now why am I so excited about this? I have been using R for many years and I care to admit but in particular the Shiny package since it first came to be I've been so passionate about it that I wanted to connect with the Shiny community via the Shiny developer series some of you in this room have been on my my show here where we have showcased just what is the potential what Shiny can bring in the world of data science in the world of analytics and of course I can't just look at all of our other contributors I want to join in on this fun too so I would even do these weird live streams sometimes of hacking together Shiny apps it might say it even gets injected into a Shiny personality it might say I am addicted to Shiny I am very enamored with this technology because it changes everything to me.
Now that's one thing in my enthusiasm about Shiny and I have used Shiny a lot in my day job and I've done a lot of pretty nifty things but sometimes it can feel like yes I'm making a difference here but I kind of yearn for more I want the industry as a whole to benefit from this as well not just in our respective silos and many years I was wondering what is the best way to have an impact that benefits everybody. My eyes opened when the rPharma conference launched I'm this is from Harvard University our first in-person event for rPharma and I saw there were like-minded individuals like me passionate about open source passionate about the intersection of Shiny and in fact some even dubbed this like a mini Shiny conference because we had so many content about the way Shiny is being used in pharma and working groups spun up from these initiatives and I thought okay this this is real now this is real how do I get involved so that's why I've been so fortunate that the rSubmissions working group and other efforts have dovetailed from this situation.
Pilot 2: Shiny in a pharma submission
So let's take a look at you know we can't just start a web assembly right away we had to figure out okay how does Shiny actually work in the context of a pharma-based submission we created this pretty fit for purpose application in pilot 2 we called it last year where it's using publicly available cdisk datasets to show what it would look like when you have these typical static like results but yet in a friendly interface they use a state of the art table generation packages and make it easy to navigate they're not 15 sifting through rtf files they are using this in one place very legible very clear and this was successful because to date before this pilot took place there hadn't been a lot of consensus on what a actual Shiny app in a submission could look like and we also wanted to explore interactivity not just the quote-unquote static stuff that give them hints of interactivity to start drilling down into the data this was successful we are very pleased that this was successful.
But with that success of course it's not always trivial to get all this set up so part of the part of the deliverable of this mock submission in pilot 2 was to tell the reviewers how do you actually get this up and running so let's look at the steps I had to put in what's called the analysis dataset reviewers guide first they have to extract the submission bundle that's going through a special gateway process they have to install R that matches the R version that we use in the development environment maybe they want RStudio so of course we encourage them they use what they're comfortable with but some packages kind of needed some compiling stuff on windows so we need to put R tools for that then we have Rm to help with package management because dependency management is critically important in this space then installing the application package library which again manifested by Rm and then finally you can run the R process it's doable we showed it could be done but that's a lot of steps what can we do to minimize this setup.
Bringing WebAssembly into the picture
Let's imagine now we bring in web assembly into this picture what in theory could those steps look like for that reviewer to get up and running well I'm still extracting the submission bundle but from there watch the application in a web process view the app in the web browser I don't know about you that seems like a heck of a lot easier than what I was showing on the previous slide about how to bootstrap all that from scratch so that is encouraging.
But what is really kind of happening under the hood so to speak when we go from that traditional method to this web assembly method and it took me a while to even grasp this myself but then I thought about more and we can think of the shiny app as the engine in a traditional sense being powered by the combination of R and shiny itself they are actually driving the application but that that is hosted on some kind of server process whether it's posit connect or a cloud-based deployment there is under the hood R and shiny are driving it but when we bring web assembly in the picture what is the actual engine it is now the user's web browser it took me a while to really grasp that but everything in that self-contained browser experience is what we had in the traditional sense but now this is different we don't need that persistent posit connect application process running here we can now let their browser do all the heavy lifting.
It's a radical shift but one that I think has immense potential here.
It's a radical shift but one that I think has immense potential here but what does it take when we look at this pilot submission this new web assembly pilot submission to do that swap of those two engines it's not always seamless but that's why we do these pilots to explore this.
First for web R there is their own binary format for packages somewhat different than what we have in the traditional CRAN repository so we have to verify what is compatible we learned through the hard way that golem the package we used to bootstrap the previous pilots app was not compatible because of a dependency it had is not being compatible so we switched to the rhino framework which had you know compatibility so we had to do a little pivoting there under the hood but once we did that the rest of the packages were working just fine and then we have of course a shiny live package you heard about this morning which is a way to help export a traditional app into the web assembly powered format we had to augment a little bit because you do a little trickery under the hood to inject the cdisk sass data sets into the web the exported application we had tremendous help from apsalon who helped partner with us on this effort to make that happen and but we want to build user-friendly wrappers into this because we want the reviewer not to have to really care about all these nuts and bolts details.
But also again we want to be where the environment is for these reviewers so on top of building all this on top of building these wrappers we rigorously tested this with custom windows virtual machines because that is where the what the reviewers are using and yeah sometimes administering r and other things on windows can be to be politely a little tricky so i needed a way to have a disposable yet reproducible way to test all this and credit to martin winpress for creating what's called cheap quick emu on linux to make that very seamless.
But what those wrappers look like we basically have a utility script it has a bunch of source functions we help them extract an application bundle and then running the app in web assembly we also gave them a choice to run the app in the traditional shiny sense if they wanted to really test the equivalents but in theory these three functions now can get them up and running with the web assembly version.
A fast-moving project
But this is a fast-moving project and i i knew ahead of time when i embarked on this journey that there would be a lot of developments and just a month ago i had things kind of ready for this but then george stagg the author of web r throws this little mastodon post and i want to highlight this specific part here dependencies and shiny live version 0.2 they can now be frozen and bundled in your app does that sound kind of familiar sounds a bit like rm right because we didn't have anything like that when we first started this pilot.
So then i see this news and because of the nature the collaborative nature of the pilot working group i got in touch with fda i said there's a big development here are you supportive of us exploring this and to their credit they say go for it because reproducibility is a huge part of why these pilots are successful so went back to the dev toolbox updated shiny live updated my utility functions and now we have the assurance that once they bootstrap this app they're gonna have that same set of packages over and over every time they run it.
Now what does this actually look like i don't want you to think that this is some radical change this is actually quite similar to the previous application that's the idea we don't want this to look like a radically refactored application this is for all intents and purposes the same app i showed you a few minutes ago it is just now the engine is different the same car it's just a different engine and our reviewers are super excited about the fact that we have the potential to ensure this environment looking like what we did last year but now we're having more smooth effort.
Interactivity and submission considerations
Now one thing i do want to mention because we're talking on the context of submission here that there is you know great power in bringing interactivity to an application now in the context of submission we have predefined objectives that all of our static results are based on and what we had in the previous version of an app was these filters that were powered by teal were actually filtering every output hence the model inferences the p values from that were also getting changed when they did a subsetting say by age or by other factors that can introduce potential for confounding and misinterpretation to the reviewers and you hear the term p hacking sometimes it is a little a little misleading when you give that power to everything our compromise and one that is an ongoing discussion as we think about the role of shiny and submissions is we are allowing that filtering within the visualization module at this time because we're not doing inference on that it's looking at that time to event profile i think this discussion will be ongoing but again these pilots helped enlighten that issue and now with the web assembly combination potentially making shiny easier to deploy we're going to have to think about these fundamentals more and more in this space.
What's ahead
So there is a bright future ahead in my opinion we are in the process of submitting this pilot for right now they just cleared it they had a good readout of a previous pilot for atom data sets created with r so now it's our turn to start the process we want to continue and investigate the features of reproducibility and also we want to make sure that we're doing the best we can to streamline execution process if you notice i showed you that we have a couple utility functions that help run that web process technically we're using r to drive that web process just because they already have it installed obviously you could do this with python you could do this other frameworks but we want to make it as easy as as possible for them to have that web process going on that machine.
I am greedy though i hope there is a future where instead of having to drive a web process for one of these other utilities that it's literally maybe it will never happen it'll be as simple as opening up a folder browsing to a locally on their chrome or edge browser and everything just works i don't know if we'll ever get there but we need some web process to start to make that execution possible and there is another component to this pilot we're using container technology in particular docker which ironically you hear great things about getting docker you know up and running not so much on their side they have a lot of challenge in their infrastructure which again was kind of a mind-blowing moment that they are actually more excited about the web assembly portion because guess what everybody's got a web browser whoever you like it or not you all got a web browser that can run web assembly container technology can be a bit finicky especially when you go the other os's.
So this has been a total team effort that's why i'm so excited about this this is not just me coming up with this random idea we've had great support from the our consortium in these pilot submission working groups we've also had tremendous help from absalon in the early stages of this pilot to look at the early versions of shiny live the early versions of web r and just what can we do to make this replicate that previous pilot as much as possible a lot a lot under the hood for making that happen and of course our colleagues at the fda who again have been on this journey with us not just reacting to us they have been proactively asking or providing their input finding the issues being open to real-time debugging sessions something we don't typically get to have so it was very very powerful to make sure that we're all on the same page with this and yeah we're not stopping now we have you know big plans in 2025 to hopefully make this even more seamless for them.
So this has been a big journey i definitely thank you for your time here you may know me more from my voice i got links to where you can find my various of media ventures but i'm here at the conference i'd love to talk to you more about this if you have questions.
And i do want to leave with a personal note why am i in this not just for the the tech of it not just for the impact of this as a whole for innovation we all have our stories right i have two very important people in my life dealing with cancer at the moment my mother my mother-in-law and i don't have visions of shiny and web assembly carrying cancer but what i do have visions of if we can speed our way of getting treatments for this horrible disease using novel technology i'm here for it so thank you very much.
If we can speed our way of getting treatments for this horrible disease using novel technology i'm here for it.
Q&A: Security concerns
Thank you eric that was really inspirational um i think we have time for one question uh was very well upvoted um so the question is about security concerns um are there any security concerns regarding that shiny app and web r and how to handle them.
That's a very good question what we what we learned is that the browser session is quite sandbox by design everything is in theory as self-contained in that sometimes even more so than what you get in the container side of it but there is an issue where we had to inject the data sets directly into that application bundle so if you do have to access data from an external source those credentials if you're hitting like an api or a database those would have to be baked into that bundle as well so if you have to access data from outside places yeah you could have security concerns there so these are all some of the fundamentals i want to keep talking about as we go forward with the potential of this technology but excellent question thank you eric thank you.