Ron Keizer | Building a Medical Device with R | Posit (2020)
The InsightRX precision dosing platform tailors in-patient drug doses to individual patients' characteristics and biomarkers, leveraging pharmacological models of drug metabolism and drug effects. These models are implemented in R, exposed through APIs, and called from a cloud-based web application
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Thanks everyone for joining. I hope this is going to be an interesting session. I'll do my best at least. So what I'm going to speak about is building a medical device with R. But before I sort of go into what a medical device is, I want to briefly explain what we actually do.
So I work at a company called InsightRX and we build a medical device as you would have guessed. But to go into what we're actually building, let's say, and let's hope it won't happen, but let's say you have to go into the hospital for maybe an infection or something, you will get a dose of a drug. But you have to realize that dose is often developed based on a sort of average of a population. And it might not actually be the best dose for you as an individual because you might be sort of on one of the extremes of that in terms of the pharmacology.
Now this is especially important for drugs like antibiotics, chemotherapy and transplantation drugs. And especially if you're very young, if you're very old or if you're very sick, this is actually very important to get the right dose for you as an individual. Now if we have some information about you, let's say you have the weight, the height, the age of you, and we know the history of all the drugs that you took. We measured the concentration of drug in your body and perhaps some other biomarkers and maybe we even have some genomic data from you. That can actually help to inform you about what you should actually receive in terms of a drug dose.
And we use that data to update our models, the pharmacology models that we use. And we use Bayesian updating to actually get to what would be the optimal choice for you as an individual. Now we wrap this framework. We built a cloud-based web application around this framework based again on those pharmacology models. So it's not like a machine learning black box that we don't know what's going on. This is all grounded in pharmacology. And the last thing I want to highlight here is that it's integrated into the clinical workflow. So the intended users here will mostly be pharmacists, but also we have a few doctors that are using the platform.
What is a medical device?
Now to get back to the medical device part, what is a medical device? Well, we all know that this and this and this is a medical device, but in this talk, I'm going to focus on what is software as a medical device. And more specifically, I'm going to talk about clinical decision support tools.
If you decide to build a medical device, most likely you will go through the following stages. We're in the pre-development stage. You basically decide which class of medical device you're going to be, what are the claims you are making. And this is also the stage where you set up a quality system. And then in the next stage, you'll begin development of your products under design controls and the quality system that you developed. And then in later stages, you'll apply the verification framework that you set up. And then before you file with the FDA, you do some validation with actual users. And then you should be good to release this into the wild.
I would say the most important thing in this whole process is probably the risk analysis that you will do. You'll do that really early on. And in this case, you scope out what are all the risks associated with the medical device. And once you have that, you can actually develop a document called the software specifications. And that is just a long list of things like the software shall be able to do this and that basically 100 times or more. So all these requirements, what you can do then is map these to actual verification steps.
Verification and testing
So what that means in practice, you actually write out a specific requirement as a scenario in code. And then you'll basically run through all these verification steps. So if you speak about the web app as a whole, what you usually use for that is, for example, a tool like Selenium. But in the context of the conference here, I'm going to speak mostly about the API, because that's the part what's written in R. And for that, mainly for the unit test, we use test that, as I'm sure a lot of you are also using.
But I want to focus on one other thing here is that what we developed in-house, which is a package called JSON2 test. And what that package specifically is doing is — I also want to refer here to the talk yesterday by Alice Huge, which was a very interesting talk, more or less about the same thing here. But going into the package where we developed, what the package does is, because if you remember all the requirements that we had to write down and to write out as code, instead of writing these all out as code, as a specific scenario to walk through, we actually record the API payloads that are being sent from the web app, and then write a document that basically specifies what we expect that API load to be. And then we compare it with the actual API output. So instead of writing all these tests, we use a package to actually do it for us. So that has helped us to implement this a lot faster than we would otherwise have been able to do.
So this package is also available in this link. It's not yet on CRAN. So that's more or less what I want to say about the validation, the verification steps that are required. But I also want to go into more about what other things are required.
So what you also find in the regulatory documents is a lot of stuff about security, privacy, stability, and things like that. But once you go speaking to clients, there will also be other requirements. Of course, clients also have a lot of requirements about security and privacy, et cetera. But they also want you to have a nice product, a good UI, UX. It has to be fast. So like I said, the end user of our products are clinicians. They won't have the time to wait for half an hour for those advice. So it has to be there very quickly.
Computations and the PKPDSim package
So then we can go into some of the choices that we made and what we developed. And the first thing here is the computations. And like I said in the introduction, we use pharmacology models. And those are often defined as differential equations. And when we started out five years ago, there was not really a package or anything in R to actually handle that. So that's why we wrote a package called PKPDSim. And the main thing that it's doing, it can do a prediction of the concentration of a specific drug in the body over time. That's basically the core of our platform. Again, this is also open source.
And the lesson that we learned here over time is that, well, at first, it took a lot of work to actually implement this ourself. But over time, we realized that this actually has been very good for us because that really helped us in the regulatory process because we know the ins and outs of this package. We know the weaknesses and the strengths. So if the FDA ever comes to a door and asks us questions about this package, it will be very easy for us to provide them with answers.
Over time, we realized that this actually has been very good for us because that really helped us in the regulatory process because we know the ins and outs of this package. We know the weaknesses and the strengths. So if the FDA ever comes to a door and asks us questions about this package, it will be very easy for us to provide them with answers.
Another thing that this package does is because we have to support not just one model, but we support over 100 pharmacology models, the PKPDSim package can actually generate new packages based on specifications that you provide. So you write a spec file where you describe the drug and you describe the equations for that specific model. That will be then put into a package example. And then that will be compiled, validated, and deployed. And the benefits of this is that it makes it a lot easier and faster to develop these models and also to maintain it and also to create the end user documentation for all these models because that's also something that we have to provide to the clinician or to the pharmacist.
Architecture and API choices
So then a little bit about the architecture. Well, we're speaking at an R conference and we're speaking about web applications. So the elephant in the room that we have to address is Shiny. And of course, I think it's fine to write a medical device in Shiny. There's absolutely no objection to that. And if you're worried about the scalability of that, you can always split off the API so that it scales independently from the web application. So that's completely fine. But really early on, we made the decision to not go with Shiny. And that was mostly because we thought in the end, the application would be too complex. It would be too hard to develop in Shiny. That's why we went with a sort of more default JavaScript stack.
If we then go into the API itself, which we of course left in R, what you have to keep in mind is the scale and the load requirements for the API. Well, we don't have the load requirements that were presented before, like a million users a day or a million API calls a day. We have a little bit less. We are growing, but we have about a few hundred API calls an hour.
But you do have to keep in mind that it's not always the same over the course of a day. We always have a peak in the morning when the users on the East Coast wake up. And what you also have to keep in mind is that the length of the API call is not always the same. We have a lot of API calls that are really fast, within a second. But there's also API calls that actually take much longer, depending on how much information you have for the specific patient.
So that also leads to the question of what API framework you're going to use. Well, when we started out about five years ago, the first API framework that we built was our own. But we quickly realized that a great application already existed, and that was called OpenCPU, which basically exposes our packages as an API. And that has worked really well for us. And we initially put that just on a server in the cloud, but later we moved it to a dockerized version, so it would scale more easily up and down, depending on the load. And that has worked really well for us.
Of course, at this meeting, there's been a lot of talk already about Plumber. We are looking into it. It has a few nice things that it can do, but there will be something for the future. We might move to that. Another thing here that would be good to mention is maybe Lambdas on Amazon, which basically allows you to upload a specific function, and then you can easily use that function, and you don't have to worry about the scaling at all. So that's another option that we might look into.
Closing thoughts
So to round off, I think, yeah, the main question medical devices are, if there was a question mark in anyone's mind, I think, yes, it's not a problem. There's no regulatory requirement that actually says you have to develop your tool in a specific language. Functional testing is really important. Remember all those requirements that you have to go through. And what really helped us, I think, to actually do that, to develop that at scale is store as much in the metadata as you can. That also really helped us with maintaining all these models as well. And then the last thing that I briefly mentioned is that, in my opinion, it really helped that we at least the crucial code for our medical device that we had to write it ourselves, so we know exactly what's going on under the hood.
Well, thank you for your attention, and I'd be happy to ask some questions.
Q&A
So we have a couple of questions, mostly on the clinical side. So how does this functionality integrate into the clinical workflow, and does it integrate into an EHR?
Yeah, that's a great one. Yes, we do integrate with all the major EHR vendors, like Epic and Cerner. And the way it usually works is the pharmacist or the doctor is actually working on a patient in their main workflow, and then they can click a button to get advice from us. And that basically brings up our web application. And then in our view, so all the specific data that we need is already there. So the user only has to look at our screen and make a decision about the dose, so they don't actually have to enter any more data.
Let's see. How about domain knowledge in the development process? Do you have pharmacists on your team? Yes, I'm a pharmacist, but I'm also a researcher and a developer, so I have more than one hat. But we do now have three pharmacists on staff as well to help with development, because they actually know what's important to actually show to the user. They help a lot out with the support as well.
So is the device effective across classes of drugs like oncological or psychotropic drugs? Yeah, we have a few modules in oncology. I would say most of our modules are in infectious diseases, because that's where the most need is. But we are looking into a lot of other classes as well. So thank you so much. So please thank our speaker.