How to build a model annotation tool with FastAPI, Quarto & Shiny for Python
Gordon Shotwell, Senior Software Engineer at Posit walks through an end-to-end machine learning workflow with Posit Team. This demo will give you a robust pattern for hosting and sharing models on Connect before you deploy them to a customer-facing system. This includes * How and why to use an API layer to serve and authenticate internal models * Why you should wrap your APIs in Python packages * Using Shiny for Python to build a data annotation app * Using Quarto to retrain models on a schedule Agenda: 11am - Quick introduction 11:02am - Demo 11:40am Q&A Session (https://youtube.com/live/zhN8IZUBCAg?feature=share) Timestamps: 1:27 - Quick overview of text classification model used in this example 2:15 - Overview of the people that will need to use the model (modellers, leadership, data team, annotators, other systems) 4:11 - Why APIs before UIs is a good rule 5:57 - What about Python packages? 8:23 - Advantages to using an API here 9:18 - Big picture overview of the workflow 11:17 - FastAPI on Posit Connect (Swagger interface) 15:55 - The way this model will be used (authorization by validating user) 19:00 - Building a delightful user experience by wrapping API in a package 25:07 - Quarto report for leadership team showing model statistics & deploying to Connect 26:34 - Retraining the model by scheduling Quarto doc on Connect 28:37 - Shiny for Python app for Annotators (people checking if model is producing correct results & helping improve the model) 35:28 - Overview / summary of this machine learning workflow Helpful links: Github: https://github.com/gshotwell/connect-e2e-model Anonymous questions: pos.it/demo-questions If you want to book a call with our team to chat more about Posit products: pos.it/chat-with-us Don't want to meet yet, but curious who else on your team is using Posit? pos.it/connect-us Machine learning workflow with R: https://solutions.posit.co/gallery/bike_predict/ In this month’s Workflows with Posit Team session (Wednesday, November 29th at 11am ET) you will learn how to use Posit Connect as an end-to-end Python platform for hosting internal machine learning models. This will give you a robust pattern for hosting and sharing models on Connect before you deploy them to a customer-facing system. This will include: 1. How and why to use an API layer to serve and authenticate internal models 2. Why you should wrap your APIs in Python packages 3. Using Shiny for Python to build a data annotation app 4. Using Quarto to retrain models on a schedule During the event, we’ll be joined by Gordon Shotwell, Senior Software Engineer at Posit who will walk us through this end-to-end machine learning workflow with Posit Team. No registration is required to attend - simply add it to your calendar using this link: pos.it/team-demo Ps. We host these end-to-end workflow demos on the last Wednesday of every month. If you ever have ideas for topics or questions about them, leave a comment below :)
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hey everybody, thanks so much for joining us today for our Workflows with Posit Teams session. If this is your first time joining us, we do host these workflows the last Wednesday of every month and so if you ever want to go back and check out the recordings, I think we've had about seven of them so far, but today you will learn from Gordon Shotwell, Senior Software Engineer at Posit, who is joining us to show how to use Posit Connect as an end-to-end Python platform for hosting internal machine learning models.
I'll be hanging around in the background here, so if you have any questions, feel free to put them into the chat. I'll also put a Slido in there where you can ask anonymously, but with that, thanks again for joining us and I'll turn it over to Gordon.
My name is Gordon Shotwell. I'm a Software Engineer at Posit and I mostly work on Shiny for Python, but before I worked at Posit, I worked in industry building tools for training, deploying, and monitoring machine learning models and I learned a couple of patterns with Posit Connect that I think are particularly useful and I wanted to go through some of those today.
So let's just get started by taking a look at this modeling code and this is a very simple, maybe probably in these days, this day and age, primitive text classification model that basically classifies text based on the words that are in that sentence to determine whether it's about electronics or not. And here we have sort of the modeling code, break the model into a training and test set, fit the model, and then print out some accuracy, some model statistics.
And the question here is, say I'm happy with this model and I want to share it with other people, how should I do that? And in particular, how should I share it with all the different groups of people that might need to interact with either this model or the data that underlies this model somehow?
Identifying the different user groups
So this is a kind of the way I would sort of break it up in terms of this sort of imaginary situation where I'm building a model, I want to share it with other people, and I can think of five groups that need to use this model. So the first group is me, like the modelers, me and the team of people who actually generated this model. Second group is leadership, who might want to get model statistics or have a sense of how this model is performing on the latest set of data.
Third one is the data team. These might be people who use code to score the text that they're encountering in their own data sets. And then finally, we have these other two groups. One is a group of annotators. This is a group of people who might be annotating data to provide me with more training data and improve the quality of my models. And there's this last group that's just kind of other systems that might come up. You know, other people using other programming languages, maybe somebody who's working on the website might need to call this model for some reason.
And all these groups have different needs and, importantly, different interfaces. So modelers and the data team, they're probably both using Python. Leadership, they probably just want to see a static site. You know, they don't need something super interactive, they just need to see something that gives them a little printout of statistics. And finally, we have these annotators, and they're probably not technical, so they're going to need more of a full-featured web app. And these other systems are going to interact with some programming language that maybe we don't know now or won't know about in the future.
APIs before UIs
One of my most sort of important principles when thinking about these types of systems is this rule, which is APIs before UIs. So when we look at this group of people, see that there are a few different groups that are using something you might call an API. It's a code-first interaction with this product. We have these Python users and these other systems, which might use some type of code. And the reason why I think APIs before UIs is such a good rule is that code interfaces are way easier to build than GUIs.
They just have fewer parameters. You don't need to worry about centering something or making the UI kind of intuitive. You can just build the code and have people interact with the code. Because they're easier to build, they're easier to change and iterate upon. This means that if you're kind of working mostly on an API, you're going to be able to make more changes more quickly to center in on the right interaction, the right data model, and the right type of user flow for your particular product.
And my experience doing this for a number of years is that a good API will usually create a good UI, both in terms of the simplicity of the code and also how intuitive it is. Once you've kind of centered on that right API, the right interaction for your problem, the GUI that serves that type of interaction is pretty obvious most of the time.
And my experience doing this for a number of years is that a good API will usually create a good UI, both in terms of the simplicity of the code and also how intuitive it is. Once you've kind of centered on that right API, the right interaction for your problem, the GUI that serves that type of interaction is pretty obvious most of the time.
The opposite is not true. I've seen a lot of times people build a GUI first. And in doing that, they kind of just sort of throw different things into some type of programming data model or some type of interface. And trying to build intuitive code interface using those same data structures can be really awkward. So if you ever tried to analyze web data as a data scientist, this is kind of why you get this big giant blob of JSON that's not particularly well structured for your purpose, right?
So if you ever have the option of building the code interface first before doing any type of GUI work, I really recommend doing this. Doesn't always happen, but in this case we do, right? We have these three groups of people are interested in code. They're also probably our earliest stakeholders, so we might want to serve them first.
Why not just a Python package?
Okay, so one way of distributing this model is to use a Python package, right? I could put this model in a Python package and have that package be installed from something like Posit Package Manager or Nexus or some other package hosting system. But this creates a few problems. And I would say there's three main problems.
The first is that packages are necessarily really hard to authenticate and authorize. So usually when you look at internal repositories that are like PyPy-like repositories, they don't have particularly sophisticated security controls on who can download those packages. But the bigger problem is that once those packages are downloaded from those repositories, they're just in the wild. There's no real way for us to control who has access to them. We can't create fine-grained controls. For example, give some people access to one Python function in the package and other people access to another one. And if the individual human who just downloaded that package shares it inappropriately or leaves their laptop open and untended, anything that's in there, including the model or any type of data that's in that package is going to be exposed, right?
So if we're working on a model that maybe includes some sensitive intellectual property or might encode some sensitive data or expose some sensitive data, that's going to be a really big problem, right? We want to have the ability to authenticate the model when it's called. And the second big problem with packages is that they always require user input to update. So if I were to distribute this model in a Python package just by itself, if I ever changed the model or updated it or changed some part of how that model is called, I would need to ask all of my downstream users to update their installation.
And they would need to do that probably in a lot of places. So they may need to update it in all the different virtual environments they have on their computer. If they deployed assets to Posit Connect, they would need to update all those. It becomes an enormous hassle. And the final reason is that that would need Python, right? Most of our users are kind of comfortable using Python, but some of them maybe aren't. Maybe I have some R users at my company. Maybe there's a Scala group that wants to call this model. And adding Python to their build environment is maybe going to cause some problems.
Hosting the model as an API on Posit Connect
An alternative, and what I'm going to choose here, is to use an API. Basically, host this model as an API on Posit Connect and have some wrapper functions that call that API. And this is going to give me a couple of major advantages. First is that I'm going to solve all three of the problems that I just expressed, right? I'm going to be able to update that model in one place by re-uploading it to Posit Connect. I'll be able to authenticate access to the API, both the top-level domain and also I can authorize any of the endpoints that people call. And finally, it's going to be language independent. Every programming language has the ability to wrap and call APIs.
So I'm going to go through kind of the overall diagram here, and then I'm going to take it piece by piece and show you the code that I developed for these projects. So this is kind of the big picture of what I'm trying to do. And I have kind of three groups here. So the leadership, I have data scientists who might use both an R or Python package, and then I have these annotators. And I'm also showing the interface that these people are going to be using. So this leadership group, they're going to be looking at a Quarto, a rendered Quarto document. These data scientists are going to be interacting through a Python or R package with an API, and then the annotators are going to be interacting with a Shiny app.
The API in turn is going to call a data store, and I'll show you what that is, but I would recommend probably this should be a database that sort of lives outside of Connect. But everything else can exist on Connect. And the reason why I really love Connect for this purpose is that it gives me a kind of safe space that I'm able to deploy all these different types of assets for all these different users without needing to have a lot of conversations with either DevOps, security, or any kind of regulatory situation at my company.
Since I have Posit Connect set up, there's this one-time investment of getting that through all the different layers of approval that you need to implement something like this. You can secure it in a way that is compliant with all of your company's policies, but once you've done that, you can just deploy these things to this product. It makes it a really useful prototyping solution both for sort of small things like a Shiny app or a Jupyter notebook, but also for these larger interconnected systems. I can validate that this whole thing works with just me, the data scientist, doing the work. I don't need to involve other people getting the infrastructure or putting that infrastructure.
Walking through the FastAPI
All right, so this is a lot. So we're going to start with this API and show you kind of the endpoint for this API, which is this API here. And so you see this API is a FastAPI, and it has a few different endpoints. So I have the ability to append training data to this dataset. I can score the model, I can update the model, I can get the last updated date, and I can query the data. And what's nice about this is that this is interactive Swagger documentation, so it'll give me kind of some information about what this is expecting.
In this case, it has no parameters, and it just gives you a successful response or gives you a string response. And I can try it out. So I can just click execute, and it's going to give me both the curl command that I used to call this, the request URL, and then also the response for this particular thing.
So I chose FastAPI for one main reason, and the main reason that... So FastAPI has a few great benefits. One is that it's very fast, but in my case, I don't particularly care about this being a fast API. It's a very simple one, so many of them are going to be fast enough. But what FastAPI gives you is this, which is a clear example of what data type this is expecting. So in this case, it's expecting a list and then a JSON object with a key value pair of text with text annotator annotation. These two need to be string, and this one needs to be a Boolean.
And the way that is produced, if I go back to my... go over to my API code here, is by creating these pydantic models that tell the API what it should expect. So here I'm able to define... I'm defining this training data class, which is just a very simple sort of key value model with these three keys and these three value types. And then when I create my Python function, I'm able to refer to that. I'm saying this data... I'm using type annotation to say this data is expecting a list of training data elements. And that list of training data elements is then going to generate this expected value.
Not only will it sort of generate that from the documentation perspective, but it's going to validate that for me. So if somebody sends a malformed... sends a different type of payload to this API, it's going to give me a 422 error. And what that means is that the API is expecting a particular form of data, and you didn't supply that data. So it's going to give you an informative error for that, without me, the programmer, needing to write any of that validation. I don't need to do anything. I just need to declare in the FastAPI type annotation that that's what I'm expecting, and then I'm going to get that result back.
So I'm not going to go through this code in too much detail. This is going to be a pretty high level sort of architectural planning thing. But one thing I did want to point out here is that I'm storing all this data on an on-disk SQLite database, and that's mostly just to keep everything on Connect without building up too many dependencies. But that particular... if you were doing this in reality, I would recommend pulling that out to another datastore.
So that's kind of... so to go through some of these models a little bit, I have append training data, which just takes a list of those training elements and sends them to... appends them to my SQLite datastore. I have a model scoring endpoint, which opens up the latest model that's been uploaded and returns a single score. So it's taking a text, a bit of text, and returning that score. I have the ability to update this model. So I have an update model endpoint, which takes both the machine learning model and the vectorizer to sort of turn your text into the right features. And finally, I have a query data endpoint, which is just sending a query and getting a result of that query, and a couple of test endpoints to just let me understand, you know, is this API working and things like that.
Authorization with Posit Connect
The way that this model is intended to be used, for the most part, is I want to allow people to... some people to upload models, I want to allow some people to append data, and I want other people to be able to score the model or query it. And when I'm doing that, I... this is kind of this notion of authorization, right? So I want to be able to sort of authenticate everybody who's joining, who's seeing this API, but I want to authorize a couple of people to do some endpoints. And I'm able to do that with this validate and validate access function here. So this function is basically asking, is user user in this control list? And if it's not, raise a 401.
And that user user is generated by this function, which is get current user, which just looks for the information in the header that's passed to Posit Connect. And what that is going to allow me to do is basically say, get the person who's accessing or viewing this API and respond with either you can access it or you can't as a 401.
So the result of that is that I have a couple of endpoints, which are available to everybody, like score model. So if you were to log open this up and try to score this score model, you would be able to do that. So you could try this out. You could say, I am a computer. I think it's probably not going to give me a very good score here. But here you see, I'm getting a 200 with the score from that model. But if you were to try to update a model or to query the data, you're going to get a 401 error.
Wrapping the API in a Python package
Okay, so that's mostly the API. And this has a couple of benefits. So the main couple of benefits we've just talked about are it allows me to authorize these features and allows me to keep my model in one place so that everybody who's using it, when they call the API is kind of keeping, getting the right up-to-date model. But it's not particularly user-friendly, right? And the main reason why it's not user-friendly is that most data scientists are not actually that comfortable with interacting with APIs, I would say, right? They don't have sort of in the back of their mind, like these are what all this HTTP status codes mean. And in order to give any kind of validation that they're calling this API correctly, you have to send something back and forth to the API, which can be a little bit difficult.
What we really want to do is we want to be able to surface errors easily and give people a lot of kind of like hand-holding to help them through calling and interacting with this API properly. Let's go back to our diagram here, right? So this API, usually when you're building APIs, you kind of want to make them predictable and pretty strict, but not that friendly. They don't, it's not that important that they give you great error messages or sort of guide you along a happy path. And in order to give that sort of hand-holding, give that sort of like ergonomics that you want when you're building a sort of delightful user experience, it's usually a good idea to kind of wrap that API in a package of some kind.
So I have my Python package here, and this package has just one class. And this is the way that I personally really like writing API wrappers, which is I have an init method that populates the top level URL and the headers. This is really useful. For example, if I wanted to change this URL, I would have it changed in one place. Or if I wanted to develop against a local running, I could set this URL to localhost. And then I typically have methods that are for each HTTP request type that I'm going to use. In this case, I'm only using post and get.
And the reason why I do this is because otherwise you kind of end up writing basically this code over and over again with all of your particular functions. So I usually have a post get. If I had other methods in my API, I would write those. And the purpose of these methods is to first use that request type to call an API with the right authentication headers and just keyword arguments, and then to handle those responses. So in this case, I want to surface basically that if it's 401, give a Python error that says you don't have access to this endpoint. And if it's not 200, just give them the error code and say, unexpected status code. It's not your fault. It's the API's fault.
So I usually start off by writing those sort of high level methods, and then write more particular methods that give people a little bit more handholding. So in this case, I have this upload data argument that uploads a pandas data frame. And what that does is, first of all, it gives you checks if it has the right columns, right? So rather than sending it and giving some HTTP error, saying like this is the wrong data type, I want to sort of stop that before I send it to the API, before I do that post. And what I'll do is I'll sort of give a check that they have these right column names, and then if they do, I will serialize that into a dictionary, and then post that to the endpoint. So this is how I would upload from code a data set of text and annotations.
I have a similar another function query data, which takes a SQL query, and just sends that SQL query and returns a data frame. And then I have two more methods. One is upload the model, which uploads, which both gives me some type annotation, that like this is the type of model, the type of vectorizer that I'm expecting. And we'll sort of open those files, or we'll like write those models to disk, and then post those two files up to the model, up to the API. And finally, I have a method to score the model, which is very simple. It just takes the score model endpoint and sends the text, and a last updated one that just will tell me the last updated time.
And the reason why this is a kind of nice user experience is that if I am then training the model, for example, I can have something like this at the top of my function here. I have just an API, I instantiate that API wrapper class. And then when I'm getting new training data, getting the up-to-date training data, I'm going to use the API query data method. So I don't really need to know that this is an API. I don't need to worry about making sure I find my key in the right place. Assuming I'm storing it in the environment variable that I'm supposed to, it'll pick it up in the right way. And I don't need to think that much about, you know, like what the HTTP response is or turning some JSON into a data frame in the right way. This will kind of handle that all for me.
The Quarto training report
Okay, so let's give this a try. So this is the Quarto documents. I'm just going to run these piece by piece. So I've got that running. And this is going to run. It's going to do a bunch of things and output some model metrics, gives me like AUC, you know, a couple of different metrics. And then I'm going to call, try uploading this data. I uploaded this data and I got a 200. So the data, the two models have been updated to the API, right? So for the data scientists, this is a really good interface. I'm able to get updated training data from this API. I'm able to train my model. I'm able to deploy my model without leaving the place that I want to, without leaving the interface that I want to use, right?
And so here in terms of where we're sitting, we have this data store, we have this API, this Python package, and that kind of serves the data scientists. If we wanted to build an R package, we would do it in more or less the same way using httr or something to wrap the API. So the next group of people is let's tackle this leadership group. And because we've used Quarto, we've actually kind of already done that. So this is a, maybe a really basic report, but it is a report that shows me, you know, what is the AUC and the various different model statistics that I might be interested in for this particular training run.
So how I could do that is I could just deploy this Quarto document. So electronics model training. This is the model training report that I'm running. So this is both training the models, exactly the code, that same code I just showed you before, and sort of assuming this is maybe like for a data science manager who might be sort of familiar with this code and want to sort of make sure that the people who are training this model are doing it correctly. And on the output, it's sort of generating a little bit of just basic scores. You can enrich this by turning it into a Quarto dashboard or adding additional plots or things like that. For this case, just keeping it very, very simple.
And this actually, whenever this code runs on Connect, it is also uploading a model, right? And since it's uploading the model, we can turn this really easily into a regular retraining service by scheduling it. So if I go over here and click schedule this output for default, and say, I want to just retrain this model monthly. So I'm going to sort of every month I'm going to retrain the model, you know, get the latest data. Maybe there's some data that's a little bit time sensitive. Time dependent, we want to have like a recently trained model serving these endpoints. I'm able to do that. So if I go ahead and save this, then this is going to run every month and generate these training statistics every month. And I can also have this email to anybody in my organization.
So that gives you that ability to sort of keep people informed and also notify them when that training has happened. If we wanted to make this a little bit more rich, for example, we could also have an error check for certain qualities of this model. So for example, I could check that this accuracy was not degrading, or say you see it was not degrading and raise an exception instead of doing the model upload. And then it would give you sort of very basic model monitoring for how your training runs were doing.
Okay, so fairly easily, we've kind of knocked out this top part of the group, right? We have this Quarto document that's running and posting the model to the API. Our leadership team can read that Quarto document. And we have a good code interface for our data scientists, which both gives them the most up-to-date model, but also authenticates them to the API in general and authorizes that particular user. So for example, if one of these data scientists had their laptop stolen, we would probably, the first thing we would do is revoke their Posit Connect key. And if we revoke that Posit Connect key, then that would mean that they wouldn't be able to access any of this modeling infrastructure. But at the same time, we have all of the nice error messages and good ergonomics of a well-designed Python package.
Building the Shiny annotation app
So let's take a look at this last group. I'm going to spend a little bit more time on the code in this one because I think Shiny is maybe a newer thing, maybe a little bit less familiar to many people. So these annotators are going to be people who are just in my company who are going to manually check that the model is producing the right result and sort of generate new training data for me. This is one of the best ways, especially for things like large language models, just improving your model. Most of the time, you're not going to be able to do that much better from the model selection or tuning perspective than the defaults. This is also true with XGBoost most of the time. If you just throw XGBoost at a tabular data set, it's going to give you a pretty good model for that data set. But if you give it a better data set, then that's where you're really going to see a lot of lift. So having the ability to build an annotation capacity that is quick and intuitive is very valuable.
In this annotation, I want this to use a web application. I want it to use a web application because I want to build that user interface to make that annotation process really fluid and fast. So let's take a look at the end results, and then we can walk through the code. So this is the annotator.
I'm just going to take a minute to warm up because I think it's starting a new process.
Or it's erroring. There we go. Okay, so just this was, I didn't, this is a spinning up a docker container, docker image, or docker container. So that's why it took a little bit of time. So this is the basic annotation interface that I'm building. We could imagine this being more complicated, but for now what it's doing is it's pulling some random data from the data set and asking me basically, is this about electronics, not electronics, or should I, do I want to just skip it? And over here, I'm getting a model score out of it, and this is letting me add some text to it. If I do that, it's going to sort of automatically recalculate this model score.
For example, I could change this to make the model score update, and it's going to allow me to update this data. Whenever I update, it's going to give me a little thing saying marked electronics, and then pull up a new one. So this kind of lets me, the annotator, get pretty quick at reading it, right? Because I can just look at this and say, okay, that's electronics, and I'm just getting the new stuff really quickly, and the UI is sort of giving me the options that I need, right? If it's something I'm not sure about, I could skip. And all of these things are referencing the API that we just built, right? So the model score is using the score endpoint. This is pulling data from the same data source, and then when I mark them as electronics or not electronics, I'm appending, and I'm adding new data to that code.
So let's take a look at this in the application. So this is the Shiny code, and the thing I want to sort of highlight is how little is actually happening on the server. I'll sort of go through this in a little bit more detail, but really all I've got here is I have this update prompt function that's being called, both when I first log on and other places, and then I'm calling the model score, right? So what this update prompt does is basically sends a query to the API to select a random record, right? And then return that string and update the text area.
And the reason why this is kind of working very well is because all of the logic is actually included in the API itself and in that Python wrapper, right? There's very little that I need to include in the Shiny app, and the Shiny app can just be there for showing the UI code and showing the interaction that I want for that code.
So this is the app. I can push this little play button in VS Code to have it run. And so what I have is I have a card with two rows and with a row and two columns, and this card is including a text area input as well as three buttons for electronics, non-electronics, and skip. And then over here I have the model score output, which is showing me the model score. How this is running is when the user first logs in, I call update prompt to update the text area. And then these reactive effects, which is how you respond to a button input with a side effect, is going to annotate the data with whatever is in that text input, and it's going to use the session user, which is the currently logged in user on Connect, to populate the annotator field and then give me a true or false respectively.
And then I'm going to show that notification and update the prompt, right? So that's going to handle annotating the data. And the model score output is going to be just a regular reactive, so whenever this changes, so without anything other than just changing it, I'm going to call the API and hit that model score. And this works well, like that's fast enough, because both the API and the app are sitting on more or less the same server, so it's really Posit Connect talking to itself, so there's not a lot of latency between the API and the server. So being able to have a kind of reactive dynamic model score that's updating all the time works fine.
And we end up with this tool here. So this is a kind of useful, you can kind of see now how this is all going to work, right? So I'm going to have some set of data annotation people at my company, maybe this is members of the data team, maybe it's like all hands, let's go annotate some data thing, log in for 20 minutes, and go just like check a bunch of electronics things to improve our model performance. I'm going to have a Quarto document that I'm using to regularly re-render that data. So whenever this scheduled Quarto document runs, it's going to pick up whatever the latest annotated data is produced from these annotators and post a new model to the API. And then everyone who's using a code-first approach is going to get access to that most up-to-date, best model that you have. And because it's all hosted on Posit Connect, it's going to be authenticated and authorized properly, and I'm not going to need to worry about security or regulatory problems with it.
Moving components off Connect
So I wanted to talk about one final benefit of taking this approach of splitting out some of your application code from a Python package or from scripts into an API, which is that it lets you move things off of Connect very easily. So for example, let's say this API became very, very popular, right? I wanted to deploy it to production. And for a variety of reasons, I don't want to put Posit Connect on my critical production path, right? There's a bunch of reasons for that. Licenses, how much licenses cost. Maybe it's a single server. I don't want to overload it. And I probably also want to not be responsible for it as a data scientist. I want somebody else to handle all the problems that come with implementing and scaling a consumer-facing API.
But because I've started out with this internal-facing API, it's very, very convenient for me to hand that off to another developer, right? So this is a FastAPI, which is a tool which can robustly and does robustly serve demanding applications. And I can say to some DevOps person, look, here, I have this FastAPI thing. Can you go put it on some sort of Kubernetes cluster to serve lots and lots of people, right? Maybe you don't want to have all those endpoints, but you want to have that score model endpoint or maybe some other ones that you're developing.
And the result is that you can develop that API internally by using this structure. And then when it's time to promote that API and send it somewhere else, you can just send that over without really any type of work. And you're sure that it's going to be correct. You're sure that it's going to handle the load that it needs to handle, work through a lot of the bugs and edge cases and are able to sort of tell them like, okay, this is what you want to do. And if we did that, then all of the other parts of our system would still work, right? We would just need to update the endpoint in this Python package, maybe redeploy the Shiny app and the Quarto document to point to the right API. But we would be able to keep working with this thing, even though we took out one part of this project and stuck it somewhere else.
In the same way, if you wanted to replace this Quarto document with a more sophisticated pipelining tool, like you wanted to use MLflow or Airflow to do model training validation reporting, you would still be able to post from those pipelines to this API to update the model or update the data store. And finally, if you wanted to host your Shiny app somewhere else, again, you'd be able to do that. So having these types of modular pieces will let you swap some of them in and out without a lot of refactoring work.
And the term there that's really useful is called an API contract, which is basically the contractual, the relationship that you're promising where these systems plug together. So like we're exposing some endpoints to all these different things. And so we're kind of signing a contract that those endpoints are going to be stable and maintain their thing, even if where it's hosted or what's happening under the hood changes dramatically. Those little points of contact between these different components of the system will all be there.
And the term there that's really useful is called an API contract, which is basically the contractual, the relationship that you're promising where these systems plug together. So like we're exposing some endpoints to all these different things. And so we're kind of signing a contract that those endpoints are going to be stable and maintain their thing, even if where it's hosted or what's happening under the hood changes dramatically.
All right. Thank you very much for your time. And I think I'm going to be able to take some questions now. Absolutely. And thank you so much, Gordon, for the great demo. And we'll jump over to Q&A here in just a second. So it should automatically push you over to the Q&A room. But if it doesn't, I'm going to share it in the chat here right now. But again, when you go over there, you can ask questions in the YouTube chat. But you can also use this link shown here on the screen for anonymous questions, too. Thank you again so much for joining us today. And I'll see you over there in just a second.
