Why Shiny for Python? - Posit PBC
Learn how Shiny for Python's design philosophy sets it apart from Streamlit, Dash, and traditional web development frameworks. With Shiny for Python out of alpha as of April, many have wondered how it stacks up against other popular alternatives. In this video, Gordon Shotwell -- developer advocate on the Shiny team at Posit -- explores the design philosophy behind Shiny for Python and how it compares to other frameworks for developing data science web applications. If you are a data scientist working mostly in Python, we hope this motivates you to take a serious look at Shiny for Python. Learn more on Posit blog, https://posit.co/blog/why-shiny-for-python/ 00:00 What is Shiny for Python? 03:05 What is Reactivity? 04:19 How does Shiny compare to Streamlit? 05:14 What are the main differences between Streamlit and Shiny for Python? 07:05 Why should I prefer Shiny over Streamlit? 09:09 How does Shiny compare to Dash? 10:22 What is the difference between a stateless and a stateful application? 12:19 Why consider Shiny for Python over Dash? 13:37 Shiny for Python's Design Philosophy
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
My name is Gordon Shotwell. I'm a software engineer on the Shiny team at Posit.
Well, we just announced that Shiny for Python is out of alpha. It's available for everyone. The API is stable. Shiny for Python is the Python version of our popular R library for building interactive web applications, mainly focused on data science web applications. It's written 100% in Python. There's no R code in the library. I think it sort of adds a lot of power and flexibility to the Python web application landscape.
So Shiny for Python is being introduced because we saw a gap in the Python web application landscape. I really felt this in my own work when I was kind of trying to transition a lot of Shiny for R apps to Python, and there wasn't really anything that I could find that did all the things that we needed the framework to do. And so Shiny for Python lets you kind of gives you a single framework that can handle a big range of complexity from very, very simple apps to very, very complicated ones without needing to learn a new thing or fundamentally change how you're developing the application.
So for the most part, the popular kind of Python web application frameworks are really narrowly focused on a single, I guess I would call it range of complexity for an application. For example, Streamlit, it's really focused on that first experience of running a web application, moving from a script that you're writing on your computer to an initial web app. But what you kind of discover as you work with it a little bit more is that the framework gets really, really difficult to use as your application gets much more complicated, which often causes people to maybe start developing something in Streamlit and then have to sort of stop and move to another framework or ask another group of people at their organization to develop a quote-unquote real web app.
And Shiny takes a different approach, which is that it gives you a set of tools or a set of concepts that can accommodate a big range of different types of complexity. So it's something that, you know, you can teach a new person in an afternoon, you can spin up a tiny Shiny app to test a statistical concept for yourself, but it can also handle very, very complicated applications, which normally you might need to use something like a JavaScript framework or Flask or Django to really accomplish. So that's the main thing it adds to the landscape is this ability to be sort of simple, but also know that they also give you the tools to accomplish really, really difficult tasks.
How reactivity works
The main strength of Shiny is reactivity, and what that means is that the only things that re-render in a Shiny app are the things that need to re-render. So it doesn't run the application code top to bottom, it just keeps track of the relationships between all the different inputs and outputs in your application and re-renders the outputs which need to re-render if an input changes.
What reactivity means is that when you're writing an application, you basically kind of decorate various different function calls which generate a relationship between all of those different function calls. So this ends up as a what's called a computation graph. And the main virtue of computational graphs is that you can minimally re-render them so the application can do the smallest amount of work that it needs to do to generate the right output. What makes this very powerful for Shiny is that if you have, for example, a multi-page app with many different components, the only components that actually cause your computer to do anything are the ones that are visible to the user. So this kind of means that you can have apps load fast initially, and then you can put expensive computations in places that people don't necessarily go to right away to speed up the overall user experience.
So reactivity hits this sweet spot where it allows you to accommodate really, really complex things, but your code doesn't really need to be that complicated because most of the work about the wiring and how those relationships are generated between the different components is really a side effect of writing the application. It's not something you need to think about that much, especially for a beginner.
So reactivity hits this sweet spot where it allows you to accommodate really, really complex things, but your code doesn't really need to be that complicated because most of the work about the wiring and how those relationships are generated between the different components is really a side effect of writing the application. It's not something you need to think about that much, especially for a beginner.
Shiny vs. Streamlit
Streamlit is a web application framework that allows you to take an existing Python script and add special function calls that turn it into interactive web application. And the main virtue of Streamlit is that it's a really natural next step from developing a Python script. So the idea behind Streamlit is you take your existing Python script, and just by adding some functions to that script, it turns it into a web application. So you don't need to reorganize your code very much initially to get something that does run as a useful interactive web application. So that initial experience is very good. The app looks very nice right out of the gate. And the main thing is that for the brand new user, they're not asked to learn very much to start with Streamlit. And that's kind of, I think, one of the main reasons why it's been such a phenomenally successful company.
The main technical difference between Shiny and Streamlit is that Streamlit re-renders the whole application code start to finish every time any input changes. Whereas Shiny only re-executes the parts of the application that need to be executed in order to display something. So this kind of comes up a lot of times when you maybe have a plot, which is based on reading in a big dataset. If you change some option that just affects how the plot looks, like the color or the scales, by default, Streamlit will just read in the whole application again. It'll start from the very beginning of the script and read the whole thing. So there's various different ways of accommodating that on a Streamlit application and avoiding that problem of just reading in everything all the time.
But all of them kind of add a lot of complexity to the application. So Streamlit apps, I would say, they start easy and end hard. So that as the app grows in complexity, the things that you have to do to get it to work become really difficult and it stops being actually an easy experience.
Shiny takes a different approach, which is that because it has this sort of reactive framework, you have to reorganize your code a little bit to get it to work initially. So that very, very first experience, you have to learn a little bit to get there. But the lessons that you learn at the beginning will carry you all the way through your application's journey, its lifecycle. So there's never a point once you kind of have a Shiny app running where the next ask, the next question becomes this monumental task that requires you to refactor your application. So the way that it accomplishes that is by executing things in a computation graph instead of re-rendering everything whenever anything changes.
Strengths of Shiny for Python
So there's a few strengths of Shiny for Python. So one of them is Shiny applications are websites. So they're simple, like at the front end, it's a simple HTML site, which means that all of the sort of HTML or JavaScript knowledge that you or your team can bring to bear on the problem can be used right away. Similarly, if you have an existing R app that has some CSS, that CSS should work, should be imported, should be importable without modification to your Shiny for Python app. Streamlit, by contrast, their way that they kind of run styling means that it's much more difficult to add truly customized user experiences on top of it. It's kind of optimized for making the initial app look really good and have some small changes that you can make. But if you really want to make a customized sort of look and feel of your application, it becomes pretty difficult.
The second main thing is the relative computational efficiency of Shiny over Streamlit. So because Streamlit has made the choice to run everything start to finish, you kind of necessarily are going to have code which is running, which doesn't strictly speaking need to run in order to display the correct output for a given set of inputs. Shiny, by contrast, minimally re-renders components, which means that you don't really need to think so hard about how your app is connected to itself or making sure that only the right things are cached. Shiny handles all of that for you. So with Streamlit, in order to build a kind of complicated app or app that reads in large data sets, you usually have to kind of manually manage callback functions or caching and caching validation, which can turn into a bit of a headache if you want to get a performant application.
So of course, all of these things to some degree are a matter of taste, like what people find difficult or not. But if you have experienced some of these problems with Streamlit, you found that execution model limiting or you found caching or callbacks confusing or fraught with bugs, I'd really encourage you to give Shiny a try. We're a new framework in this world, but we think we do have something really special to offer and we'd love your feedback on it.
Shiny vs. Dash
So Dash is a slightly older framework in the Python ecosystem, a web application framework that's built around this idea of stateless functions, which means that each callback or each thing that's creating a graph or a table is independent of all the other pieces. And its main goal, and it does, I think, a remarkably good job at accomplishing this goal, is to build applications which are really, really efficient to deploy. So I would say a lot of the things that Dash asks you to do as a developer are there so that the deployment of a Dash app and the way it's served by servers to the user is really, really efficient. And that's kind of the main thing that it optimizes for. The value of that is that it's easy to scale apps horizontally by adding a lot more servers, a lot more processes, and because you have these kind of independent callbacks. When you have done the work to develop a good Dash app, you've gone through that process, the app that you end up with is really, really efficient to serve to users. So the kind of user experience is fast, and most importantly, the process of scaling that app is really, really easy for a DevOps person.
So stateful versus stateless. So stateful apps are apps where the server is kind of maintaining some information about the user session. Mostly when you're working on a script on your computer, you have some kind of global variable or information that is passed, that is held in memory on your computer and passed to a bunch of different functions. So in stateful applications, that information is held in memory by the server, and it's passed to the individual function calls. Stateless applications take a different approach, which is that the server is not maintaining any kind of state about the user, like each component of the app kind of needs to go from start to finish to deliver that particular graph. So if you have three graphs or three different components being rendered on a website in a stateless framework, they might each read in a data set, each read in the same data set in order to generate that component. And the value of that is that multithreading becomes really, really easy because you don't need to keep any of those components in sync.
The issue with developing stateless applications is that it's not how you usually program. So if you're programming on your computer, you usually are dealing with one process on one laptop, one computer. And as a result, it's really natural to use variables to store information in server state and to have functions read data in one place and have it be used in many places by rendering functions. And so when you're moving from that sort of stateful way that you've been developing a project to a stateless dash application, it can be really painful, and you end up needing to do quite a lot of work to move your process into that world in a kind of reasonably efficient way.
When you compare Shiny to Dash, the main thing that Shiny has is just a richer set of server states. Shiny is stateful, and what that means is that you can read data in one place, you can have reactive calculation that does some work on it, and then pass the result of that reactive calculation to as many different rendering functions as you want. So it really does a better job of respecting the kind of do-not-repeat-yourself principle of computer programming, where you're reading the data in one place, you're processing it in another function, and passing it to rendering functions only afterwards.
With Dash, you kind of are forced, if you have like an application with three graphs, you have a couple of choices that you can make. You can have three functions which each read and process the data, or you can have one sort of mega function that reads in the data and then updates all three plots. And both of them are kind of a little awkward because that's not usually how you're used to developing applications. Usually what you want to do is have each function do one thing and do it well, and you want to have each calculation happen in one place. You don't want to have that code be repeated anywhere in your application. And when you're moving to Dash, you kind of have to pick between a few options, none of which are terribly appealing.
Design philosophy
So whenever I've developed a web application, usually I need to start developing it very, very quickly. I have almost no time to get a minimal thing out the door. But it never stays simple. It always gets more complicated. There's another ask, there's another requirement from some user, and the application slowly becomes more complicated. It's used by more people, more people want more things from it. So our design philosophy for Shiny is that we want Shiny to support any of those asks. So even though it's something that you can start, get something up in an afternoon or a couple of hours, you know that you have all the tools to make it a business-critical web application that's used by many, many people. And we don't want you to end up in a place where there's some task, something you really want this application to do, and the framework just won't let you do it.
So that's kind of what we mean when we say that it strikes a great balance between simplicity and complexity. It starts simple, but that application can grow to accomplish really, really complicated problems without needing to sort of fight the framework.
So that's kind of what we mean when we say that it strikes a great balance between simplicity and complexity. It starts simple, but that application can grow to accomplish really, really complicated problems without needing to sort of fight the framework.
To sum up, most of the popular Python web application frameworks are focused on a kind of narrow band of problem complexity. Streamlit is focused on your initial application just right when you start. Dash is focused on applications which are where deployment efficiency is really, really important. There's not really a single framework that lets you grow from very, very simple problems to very, very complicated ones. The killer feature of Shiny is that it does support that type of complexity. So it's a simple set of principles to learn about building your initial Shiny app, but once you learn them, you don't really need to change them as your problem gets more complicated. It supports a full range of CSS and JavaScript customization and also has an execution model that's efficient enough to handle data science web applications, which can often be quite computationally intensive.
Our main hope is that you all try this framework, let us know what works and what doesn't, show us what you're building with it, and help us develop some of the community contributions that we really need to make this a successful project. So give it a try and we can't wait to see what you build with it.