How to combine Matplotlib, Plotly, Seaborn, & more in a single Python Dashboard! (Shiny for Python)

Transcript#

This transcript was generated automatically and may contain errors.

Hey what's up everyone and welcome back to another video. In this video we're going to see how we can combine Matplotlib, Plotly, Seaborn, & all of your other favorite Python visualization libraries into a Shiny for Python app. One of the things that I love about Shiny is that you have this capability. I feel like so many times in the past I've done some analysis in Python and then been working for a client, working for a company, that requires me to port that into something very specific not using Python. Whereas if I had Shiny for Python you know five years ago I could have easily just spun up a dashboard using all the same visualizations.

As a reminder in the last video in this series we built a dashboard that looked like this with some placeholders where we can put different Python visualizations. In this video we're going to fill in those placeholders with like a heat map you know another type of heat map of the actual US and then change this top graph to use Altair instead of Plotly. And throughout this process we'll learn about all the different kind of annotations and how you can incorporate many more libraries than just what we cover in this video. And then finally we're ultimately working towards a dashboard that looks like this by the end of the series. I think the only thing we might not cover in this video is really the styling and the coloring and some of these final tweaks. So we might not have all these colors set by the end of this video but we'll have all the graphs and we'll leave all the styling details for video five. Should be a lot of fun.

Getting started with the code

So to get started with the code I recommend going to this GitHub URL. I'll link it in the description and depending on what part you want to jump in on you can scroll down and see the different parts of the series. We're going to be building off of the source code for part three in this video so you can go ahead and look at the code right here or if you want to specifically have all this code locally what you could do is you could fork the repo so that you have your own personal copy of it. Once it's forked in your personal copy you're going to click code you're going to copy this URL open up the terminal window and you're going to want to do git clone and paste in that link. You're going to navigate into that folder and then you want to get all the branches in that repo. You're going to do git fetch origin and then finally you can do git checkout video three and it will switch to the branch that has all the code from video number three.

You want to navigate into the sales folder. You might need to install if we look in that folder there's requirements.txt. You might need to install the requirements.txt by doing pip3 install-r requirements.txt and then the final step that you might want to do if you use VS Code you might have access to this code period to just directly open that from your terminal and we see we have our app.py right here and I can click run and the dashboard loads right there and we can start working on this code. You might even create another branch that's separate from this code however you want to approach it but that's my general recommendation.

Rendering Matplotlib and Seaborn with render.plot

First we'll quickly want to change this to video four of five but after that I would say a good starting point would be to really understand how to render matplotlib visualizations using Shiny and the kind of keyword for that is this render.plot function. So how about we start down here with the heat map and we'll start by just doing a simple matplotlib plot but then we'll make a heat map using Seaborn and ultimately there's a lot of Python graphic libraries that are all based off of matplotlib. So even though this render.plot really focuses on matplotlib it covers so many different libraries in Python because so many are based off of matplotlib.

So we can go down here to this heat map and we could start plotting a chart down there using matplotlib. So to do that we'd want to add a render.plot annotation here. We'll want to pass in a function so I'll call this plot sales by time how about and then in that we'll pass in our matplotlib code. So as a reminder if you're using matplotlib you can import it by doing import matplotlib.pyplot as plt that's typically how you'll see it. Down here we could immediately plot something with matplotlib by doing like plot dot how about bar chart. We could pass in some random numbers for our x pass in some random numbers for our y and that will actually work right there.

So even though this render.plot really focuses on matplotlib it covers so many different libraries in Python because so many are based off of matplotlib.

As you can see we have a little chart but let's make this actually use our data. So as a reminder we're using sales data we have a bunch of different products how many times they were ordered the price of the order and when they were ordered so this is like a unix timestamp but ultimately we can convert that to a date time object and get more easy like months and hours from it. So if we want to do sales by time and I'm thinking of time of day so what hour from 0 to 24 were these purchases made well to do that we might want to add another column. So we're accessing our data frame using this dat reactive calc that's over here at the top of our code found right here.

I'm going to add another column to that I'm just going to call this hour and hour is going to be equal to df order date our date time object dot dt dot hour we can easily access a numeric integer hour from that. If we go down to our data frame just go to the right we see we have an hour now 14 20 18 that all makes sense. I think the max actually is going to be 23 so if you wanted to like offset this by one you could but I think it actually makes sense starting at 0 going to 23 because ultimately once you get to 24 turns the next day so you don't actually want to see 24 here in our data.

And then we can access a simple bar chart by doing something like sales by time or sales by hour how about equals df dot value count or df how about hour and then grab the value counts of that. If we printed sales by hour and rerun this or actually just open our terminal we see we get something that looks like this that has the hour and then the count. The last step to make this a little bit easier for us I would recommend doing a reset index and then watch what it prints out now it prints out a nice data frame with hour and count.

So we could easily pass in a x equals sales by hour hour and then a y which would equal or I guess they use height now these days but I think if you pass in y it's fine too sales by hour count. We see we get something that looks like this and you could pass in like your x ticks different uh matplotlib keyword arguments. So our x ticks might be like numpy do we have numpy imported I guess we don't so I'm going to import that as well.

You could do something like np.arange 0 to 24 I think that would give us the right amount of ticks. So that gives us all the tick marks that we need that's a simple matplotlib bar chart you could obviously customize this as you need it knows to output that plot object but you could also return a figure here and it still would work.

But we actually want a heat map using Seaborn so we can also do that within this render.plot function. If you go to Seaborn the documentation you'll see that it's a python data visualization library based on matplotlib and there's a lot of other matplotlib based or matplotlib adjacent libraries that we can use. Some immediately that I know about are like plot9 you can use pandas directly there's a lot of pandas.plot functionality that you can immediately tap into.

But what would it look like to do a Seaborn heat map like we have in the ultimately the final plot that we're working towards. To do that yeah go to the Seaborn documentation maybe look at api look at somewhere in here probably have a heat map heat map and look at how it's done. We have a pretty simple heat map that we want to deal with we want to just have basically a one day array but I think that only takes in two days so we kind of have to play around with it a little bit. So I'm going to also import Seaborn.

And we can change our chart to do something like probably want to keep the sales by hour we look at our docs basically takes in that 2d array so we just need to format our values to be this 2d shape. We could do that in here by switching this up a little bit so instead of resetting index I'm going to actually re-index it I'm going to pass in that np.arange 0 to 24 basically set this and sort it to be between 0 and 24 but use that as the index. So no longer is like our own separate column we have that we also as like good practice with re-index you should probably have a fill value so if ever you don't have one of those numbers we want to have that still show up in the heat map so it would just be zero in this situation.

We're counting a lot of product sales so we should always have greater than zero value but this is just good practice to do. What else do we need to do here I think ultimately we might get errors if we try to do the heat map by just passing this sales by hour I think it expects it might might take a day. I think we're going to have to reshape this. Reshape we want it one column and then however many rows we need which would be like 24 so if I could reshape this is 24 to 1 it should work I guess we got to do sales by hour dot values dot reshape to turn into numpy and then numpy accepts.

Nice that's our sales by hour I don't love these colors and I would love I'm trying to think of other things if we look at our final visualization we have like the numbers right inside of the heat map and I think that looks kind of good so I'll pass in some parameters to this to give it more of that look. So annotation equals true and honestly there's a few different ways you can look into the documentation to see these different options you can also honestly use like a large language model like chat gpt or something to help you get it exactly how you want it.

Didn't need any x tick labels zero there's kind of unnecessary y tick labels it's kind of nice to have more of an hour look so you could do a list comprehension here which just do something like i colon zero zero for i in range 24. Oh wow that's looking a lot more like it I think the only other thing we would want to add is some labels and you can actually use the same from that matplotlib labels to fix this so plot.title might be something like number of orders by hour of day the x label will probably be hour of day and the y label would just be order count. That got a nice heat map there perfect.

That looks good and you see all this it knows to render the plot properly and as we see we're editing some matplotlib things but we ultimately use Seaborn and we get that chart there so you have access to all of Seaborn and you'll use the render.plot for that.

IPython widgets: render.widget and render_altair

Now what are our other options here so the other big category of things that we might plot with Shiny are ipython widgets. So if you go to this documentation page I'll link it in the description we can see that there's all sorts of ways to plot jupyter widgets within Shiny. So some that show up right out of the box here are Altair Bokeh Plotly pydec etc and what you'll see if you read into this documentation is they all kind of are under this subcategory render widget but kind of give some fancy special things for specific libraries that Posit has specifically built support for like Altair Bokeh Plotly and pydec. You'll see that instead of using the render widget annotation for some of these you can directly use a annotation that's specific to that library like render plotly here.

So to do that we need to import render plotly from shiny widgets and then we can kind of leverage Plotly as we expect and we've already actually seen this in our code so if I go back to our VS Code we see that we have render plotly already being accessed here. So you can kind of keep that as is but let's say instead of using Plotly here for this first graph so we'll see this first graph if I make it a little bit bigger how about we switch that to use render altair so the Altair python library. How would we change it to allow us to do that well the first thing we need to do is also import render altair it's the first step we might also just import render widget if there's other extra cases that we want to deal with here.

Okay and now how do we convert Plotly to render altair honestly what I might try is seeing what our good old friend chat gpt or whatever your llm of choice recommends for this so I might type in convert this Plotly code into Altair in Python.

Okay and I'm curious if I just copied this code and I pasted it in here we're going to have success well we're not going to have success immediately because we're not even importing Altair so import altair as alt. Then it is saying sales over time altair missing two positional oh this is not necessary to pass in because we already know it I guess we would have to grab df equals dot then our city is input dot city it's kind of confused by the uh Shiny stuff. Look at that we already got a nice looking uh Altair chart.

Oh this is rendered plotly it actually works just fine it's just the type of thing is if in the future something changes and this needs to be very custom for Plotly you might run into errors here so you'd want to actually use the annotation render altair. So if there's a specific annotation for the library that you're using go ahead and use it but if you don't see one for your ipython widget just use render widget and that will kind of work for most situations that you need. We can see that if we change this it does update we kind of get good results overall that's pretty cool to see.

So other than Altair Plotly Bokeh pydec what other options do you have with ipython widgets well this is a list of different visualization jupyter widgets that are available so if you see any of these that you particularly like you're probably in a good spot to use the render widget function. I don't know if these have all been tested with Shiny but I think that this is you know something to explore a bit further. I'll link the list of these jupyter or these python widgets in the description but also know that if you go into other you'll see that this is actually also linked in the documentation. One other one I would worth what's worth mentioning is like ipy leaflet I think it's pretty cool to see so we can see like a map like this is rendered via the render widget annotation.

Adding a Folium heat map with render.ui

Alright what else what else what else what else alright we've done two of the changes we got our heat map and we got our Altair chart here I think the last thing we want to do is maybe do a sales by location heat map and the ipy leaflet is one way to approach this. I feel like for heat maps one library that I have particularly liked is the folium library so I think it's worth discussing the folium library a bit. So our code for this placeholder is right here sales by location map and how can we use the folium library well the first thing we need to do and I might even just pull in something for this to folium heat map example.

Okay so we ultimately need a data frame that has latitude longitude and a weight value that's the three things that we need that's basically what we want to replicate. Okay so to import folium import folium like this all these should be in the requirements.txt so you should already have these if you don't just do a pip install of these different libraries. Okay if we look at our data frame we actually already have latitude and longitude there so we basically just need the counts the counts for those latitudes and longitudes to figure out our sales locations.

So our counts in this situation would be the quantity ordered and the heat map will be smart enough to find if you have similar latitudes and longitudes and similar quantity or in quantity orders by each other ultimately you know your heat map will show those red zones. What we'll probably want to do is just go ahead and take our how about we call this def plot us heat map we would want to get our data frame which is equal to the reactive calc and then maybe we have a smaller data frame that's just basically like heat map data equals data frame latitude longitude and quantity ordered.

You might want to get the values off of that let's print the heat map data real quick and a good exercise would be to try to get this heat map working on your own before you see the final code. Now we see the data that looks good that looks like heat map data so from folium if I go back to that the first thing we wanted to do was build a map so that might look something like this this is a latitude longitude start location you kind of have to play around with this I would say to figure out what works best but it's a good starting point. Then we need to add data to that map well we can to do that we can do heat map and we can pass in the heat map data and we want to add that to our map and then ultimately we want to return our map.

And so let's see what happens when we run this it looks like we have some sort of error and it tells us that it's having trouble turning it into an ipy widget and it's instead recommends using Shiny's render ui decorator so let's go ahead and try that render.ui. And wow we got the location map work and let's go.

I think one thing that's interesting is folium versus ipython leaflet this ultimately gives you a object that has like html source code all kind of built into it if you printed map dot representation html you'd actually see all the html associated with this map get printed out and I think as a result of that like we can easily render this html. render.ui is like the keyword the annotation to use whenever you need to render html and so something about this folium library isn't friendly with ipython widgets but it does have this html representation so if you ever need a render html within your Shiny apps render.ui is probably going to be what you need.

render.ui is like the keyword the annotation to use whenever you need to render html and so something about this folium library isn't friendly with ipython widgets but it does have this html representation so if you ever need a render html within your Shiny apps render.ui is probably going to be what you need.