Transcript#

This transcript was generated automatically and may contain errors.

I am so excited to introduce our featured lab manager for today, Nicola Rennie. Nicola, would you like to say hello and introduce yourself?

I can. Hello, I'm Nicola. I do a lot of data viz stuff. I've been using R for about 10 years, and I'm very excited to be doing some TidyTuesday visualizations live today.

Yes, I'm so excited. Nicola is going to show us her process of how she works through a TidyTuesday visualization. But we only have an hour here. And quite obviously, she can't go through her entire process, but she is going to be sharing some of the ways that she thinks, some of the tools she's developed over the years. She'll talk about how long she's been doing this, how long she spends on stuff like this. So get your question asking, typing fingers ready. All right, take it away, Nicola.

Introduction to TidyTuesday

Awesome. So yeah, if you don't know what TidyTuesday is already, it's a super awesome sort of resource. So basically, a new dataset is released every week, every Tuesday. Although sometimes it's a Sunday, sometimes it's a Monday, you never know. And the idea is that you make something with it and share it and share your code with other people. I started doing TidyTuesday. It was my COVID lockdown 2021 project. Other people took up baking or cycling or whatever. For some reason, I got into TidyTuesday.

So there's a GitHub repo. And if you scroll down in the GitHub repo, you can see all the datasets from each week. So this week, the dataset is on Twin Cities. So if you go into the readme for a specific week, you get a little bit of information about what the data is about, where it's from. You get some instructions for how to load it into R, Python, Julia, and their CSV files, if you really want those. And there's some really nice data dictionaries that explain what all the variable names mean. And those are really, really helpful.

So this week's data is all about Twin Cities, so links between different cities around the world. So there's actually two datasets this week. So one of them is all about the actual cities. So you've got an ID, the name of the city, coordinates of where it is, what country it's in, and continent. And then the second dataset is the actual links between them. So it's just two columns and start and end point of the links. So I'm going to jump over to R and see if we can start making some things.

Template files and workflow setup

So I have all of my tidy Tuesday visualizations from the last five years all live in one GitHub repo, which is fairly nicely structured.

One of the things that I started making to try and help save myself some time was some template files. Because when you're doing, whether it's visualizations or analysis or whatever it is, you end up doing the same thing or very similar things week after week. And I didn't want to just keep typing out the same things every week. So I made myself a little R package, which has some templates in it. So I have this use tidy Tuesday template. And what I just need to put in here is the date. So if I run that, that creates a whole bunch of files for me. It creates an R script and a readme file.

So we load all the packages to start off with, basically the entire tidyverse. I also use the show text package for text. I love the show text package because you can load like Google fonts, which means you never have to worry about trying to install fonts on your own laptop or try to give other people instructions for loading fonts on their laptop. Also got a couple of packages for doing some fancy text. So like some bold highlighting or like color text. We've got glue for sticking together variables and text. And we've got the ggview package, which is super useful for previewing your plots at the same size and resolution that you want to save them at. So if you've ever done that thing where you're like spending ages tinkering with your plot and like making it look really pretty and perfect, and then you use ggsave and it just like, it's a complete mess and it looks nothing like the preview you had in RStudio, ggview is really good for that and I'll show you how it works.

So if you've ever done that thing where you're like spending ages tinkering with your plot and like making it look really pretty and perfect, and then you use ggsave and it just like, it's a complete mess and it looks nothing like the preview you had in RStudio, ggview is really good for that and I'll show you how it works.

So the other sections in this template are basically like to load in the data and the templates files are put in the correct date and stuff. This is loading some fonts from the show text package. I've got some defaults in there so that if I'm like having a busy day and I don't have time to pick out fancy fonts, I've got something that I know looks all right. I've got some default colors. Again, if I'm busy then I'll use those. I've got some sort of like setup because every time I'm making a chart it's like almost always got title, subtitle, caption and all of that. So just like prep those variables ahead of time.

And then I've got kind of like a theme that's set up and it's all kind of written out rather than being like in a personalized theme function so that it's easier to edit one bit of it for a particular chart. And then I've got some code at the bottom to save it in the correct place so I never need to worry about figuring out where I'm supposed to be saving stuff.

Fonts: show text vs systemfonts

Do show text and systemfonts do different things? He says because fonts and ggplot2 scare me and honestly same.

So they can do the same thing but they can also do different things. So systemfonts is really good for finding fonts that you have installed on your computer. That works great if you are just making it yourself but if you're collaborating with other people and one of you is on Windows and one of you is on Linux and you're trying to use some Microsoft font, then you have to give instructions for how you install fonts. Whereas this sort of font at Google, I mean as long as you have an internet connection it just sort of downloads it from Google so you don't have to worry about whether you have that font, whether your collaborator has that font. It just kind of loads it in for you. You can load local fonts with show text as well if you do have something special or different that you want to use. It's kind of incentivizing to stick with Google fonts though because then you kind of can't get it wrong, right? Yeah, the only time I've like it's not worked is when I've been on a train and there's been like no wi-fi and your script doesn't run because you can't connect to the internet. But that's like a weird, it's a very niche problem I think.

Loading the data and choosing a chart type

So the only thing I'm going to grab from the R for Data Science tidy Tuesday repo is this little bit of code here to kind of load in the specific data sets. And then now I have two data sets. I have one called cities, right? So it's got that like unique ID, the name of the city, coordinates, country and country code, and continent. And links looks a little bit confusing but it's just the ID for different countries and source and target just says that there's a link between those two countries.

So I'm also just gonna run this in the background and then kind of start to think about what on earth I'm gonna plot with this data. It is a really nice data set, like it's not super big like in terms of number of rows or columns but there's so many things I kind of want to look at with this data. So one of the things I would quite like to look at is whether you can connect every country in the world through like one giant loop of like these twin cities. Other things you could look at is like which country is the most connected. So you could do some sort of like global map in color based on how connected they are. What I kind of want to look at is how like connected the world is to each other in terms of how close things are. So like is a city in Europe more likely to be connected to other cities in Europe? So what I'm gonna, my brain's sort of thinking in terms of chart type is something maybe like a Sankey diagram or a chord diagram.

But then you kind of have to choose what level you do it at. If you do that at the country level it's just, I think it's just going to look like a big pile of spaghetti because there's too many countries and there's just going to be lots and lots of lines. At the same time I think if you do it a continent level there's like, that might work. There's like, I think there's six in the data set. So that will probably work okay. I think what I'm going to do though is pick one city and then sort of look at how connected that is.

Okay so I'm gonna have a little look first of all at what like list of cities we have. I think like when I'm doing tidy Tuesday quite a lot I quite often pick like cities or countries that I know like are some way connected to me personally because then you find out like interesting things about like where you live or that kind of thing. So I want to see if I start with cities and then filter the country code to be GB. I quite often pipe that into view at the end. So you get this nice kind of human readable look and then I can browse through all the cities in the UK that we might focus on. I'm going to go Edinburgh because that's where I live. So purely self-serving.

So that's giving me one row of the data and the thing that I'm really interested in then is going to be this ID column. I think what I'm also thinking at the moment is this is something that I would quite like to parameterise. So the idea is that you can type in whatever city you want and at the end you will get the plot for your chosen city.

Wrangling the data

So I've got my ID. So what I need to do now is go into this links data set and pick out like all the links that relate to this city ID. This one's a little bit weird because the order of the source and the target doesn't matter at all. So I need to filter it when either the source is equal to the city ID or when the target is equal to the city ID. All right so I've got 11 links and you can see that like some of them are in the first column and some of them are in the second column and that's um it's like a little bit annoying.

So I want to get the ID. Let's do if else. So if the source ID is equal to my city ID then what I actually want is the other one and then vice versa. Okay so now yeah now this column is all of the like IDs of the other cities that it's linked with and that's the only thing I actually want. I use select all the time to like tidy up because I really hate having extra columns um that I'm not interested in kind of hanging around in my data. Purely personal choice. But now I can join this with the city's data by the ID right and that's my sort of like I guess tidy final data set. So I have 11 cities that are all linked to my chosen city Edinburgh and I've got like all the information on them.

So I always save this as like plot data. It's not the most imaginative name but I do it every time.

Building the sunburst chart

I think what I really want here is like a sunburst diagram. So like in the middle you're going to have that chosen city and then you will have all the cities it's connected to then all of the countries they're in and then all of the continents they're in. I'm kind of thinking if you colour by continent then you can see like quite clearly like which continent is going to dominate and whether that's the same as the continent that your chosen city is in. So sunburst diagrams are like kind of like I guess like a circular tree map and or kind of like stacked doughnut charts.

So one of the things I quite often do at this point is if I'm thinking about a chart that I know I've made before I quite often like go and look at the code for that chart and just like copy and paste it and then edit it rather than trying to write it from scratch every time.

So I have a little Shiny app that grabs all of my tidy Tuesday plots from that repository and you can kind of search by package. You can also search by name which I appreciate is not the most useful. But I remember doing a sunburst chart on Australian Frogs for previous tidy Tuesday.

So this is what sunburst chart is and I have links to the code. So when I click on this it just opens and then I get the code. Oh my gosh the benefits of good documentation for your own stuff. You have to learn the hard way I think. I still curse 2021 me for their terrible file naming system.

But yeah so this I can see like how did I make it the last time and the last time I made it in a really weird way. So normally if I was making a sunburst diagram I would probably use the ggforce package which has like geom arc bar. But I wanted to make it interactive so it had these little tooltips and that doesn't work with the giraffe package. So I think I remade instead of arcs I made them all as polygons.

So this one was a little while ago and I think it was before a newer version of ggplot2 came out. So I think previously if you wanted to make like pie charts or donut charts in ggplot2 you could make like a bar chart and then use a coord polar which kind of like pushes it into a circle. The problem with that is that like putting annotations and stuff on it is a nightmare because their coordinate system just goes crazy and you don't know where any points actually are. But the newer versions of ggplot2 have like a coord radial which I think is basically just fixes that issue.

So what do I want? First of all I need to figure out for every given- so I'm gonna have three layers on my radial chart, my sunburst chart. I'm gonna have like cities, countries and then continents. So I need to take my plot data and then group it by one of the regions. So I want to count up the number of each city which okay there should just be one.

So I then did that and then I need to keep the city the number and because I want to color by continent I need to keep that in the data. Now I want to do the same code for country and continent so I'm gonna stick this as a function to save me wrangling lots of little data sets.

So yeah I've got this little function here which takes like a region level so that will take city country or continent as an input and it will return me a data set. You may or may not have seen this like weird curly bracket notation before. It's like a weird sort of tidy eval thing that lets you put variable names in without quotation marks essentially. Yeah I call it a double hug.

So if I do something like region data of country I get that right. So I still have the country the number and if I do continent I think it's gonna give me some duplicates. So let's stick distinct in there. I don't want to count them all twice. Okay. So that's a nice quick way of getting my data sets.

So start actually making a chart. So I want to do geom col. So my data is going to be this region data. I'm going to start with city and then mapping is going to be a little bit weird. I think it's not going to do what I want. Okay. That sort of looks correct. And then if I add coord radial. Okay. It's got a weird gap here which I don't want. I always use like expand equals false in like most of my charts because I don't like the weird extra space. Okay. This is kind of made a pie chart. We made a pie chart by accident.

Okay. But this this should go away. So if I switch y to 2 that should stack another one on top of it. And I should have region data is equal to country. Okay. This is looking better. And then if I do one more this is going to be like continent and move that up to three. Okay. That starting to look better.

Now we're getting closer to like a dartboard. Oh but you know what I saw we had a question that might benefit us to do well before we get too far away from your dplyr query. And it was from Nathan saying can you say why you used mutate rather than summarize. Yes. Because I want to keep continent in there. So if I normally only wanted to like when you summarize it will get rid of everything that you haven't kind of grouped by. But yeah I'm using mutate because I want to color by continent. So it needs to stick around. So that's the only reason I'm kind of using mutate there.

All right. Okay. So I want to get rid of this pie chart thing in the middle. So I want to do here is if I filter the city's data to get the city chosen city. So and then the X value is going to be all of them. So it's within number of rows. All right. That sort of looks. What I mean like is a rough idea. One thing that's a little bit weird is that you can't see like the different cities because there's no outlines. So I'm gonna if I say color is equal to my text color. Does this help. Yeah. So now it's a little bit easier to see that like you have your one city that you picked in the middle.

There's like three four seven cities in seven countries that are all in Europe which means we now get to do the fun part which is like making it look pretty.

I think it's great to see that you still have all of the like random bits that are around right. Like you haven't like theme voided anything. Yeah. I quite often like theme void is great for making kind of charts like this in maps but quite often I leave it to the very end because it can really help with like positioning annotations and things if you can be like I want to go there and you can see what those coordinates are. And if you are working with a grid I made the mistake one time in a very big project of just sort of theme voiding from the beginning and not showing my indexes basically on the side and I couldn't figure out why my plot was upside down.

Applying the template theme and ggview

So this I guess this all sometimes feels a little bit like cheating when you use a template so you can just stick a plus on the end and then fire in all of the sort of template like theme stuff that I had. So I tend to just run it and see what it looks like.

One thing that you might notice one I guess the text is like slightly different size and also it's opened in this viewer tab and that's because it has this plus canvas thing on the end and that's from ggview. So here you can say I want it to be like 7 by 5 dpi is 300 which matches with ggsave and set the background color of like the whole area which means that when like if you use ggsave to save this that is what it will look like. It shows you exactly as it will be and it's like a quite nice way to play around with some different sizes.

So I think actually for this chart I probably want it to be portrait. If I put some text above it and get rid of that legend. I'm gonna do that just now. Legend.position equals none.

Okay yeah all right. I am gonna switch this to theme void to get rid of all of the stuff I don't want and I'm gonna add an annotation to put like the name of the chosen city right in the middle.

Colored text subtitles and captions

Okay so it's not super in the middle but it's that's all right. Okay what else do I want to do? So one thing I want to do is I want to get rid of the little white gaps between them. A little bit annoying. So in ggplot2 the width of columns is 0.9 not one.

I don't like the ggplot2 default colors. I quite often use rcartocolor because they have this kind of like nice combination of like being accessible and being quite good colors but also looking quite nice. So I use them quite a lot and I'm going to define a color palette from there. The other thing I want to do is like map continents to colors kind of ahead of time because you can see there's only four in this chart but there's actually six values. So if you were to choose a different city then the colors might switch order which is not great.

What I quite often do with kind of like instead of a legend is I use like color text in the legend so you'd have like a nice sentence that writes out all the different continent names that are and the color that applies in the chart. So to do that you can use I think you can use marquee but I've been kind of using ggtext for so long that that is kind of my default. So the way you do that is with like writing spans so you put like the name inside the spans and then style it.

Because I want to map over everything I'm going to use purrr so I have this like list of continents and this kind of like list of color palettes and basically for every element in color palette I want to create like some html text with the name of the color and the name of the continent. So I can use glue to do that and I take my spans from here so inside the span you have style is equal to so the color is going to be the color palette.

So this gives you like this factor so you have this span and you're like okay asia but in this color so what I want to do is take that and like construct a sentence with it. I mean you can write out the sentence and pull those values in but you can also use like string flatten from stringr so you can say like between each element put a comma. But you can change the last one which is like a really nice feature for writing sentences. So that creates this it looks horrendous which is why you use things like purrr so you don't have to type all this out yourself. Um but you get like oceania and south america um and then.

So this is my subtitle text so that's gonna be my subtitle colors and then subtitle can be something like um like twinned cities in my long list and then full stop. Okay. And then I can stick that in to subtitle with labs so subtitle equals st. Okay you can't see all of them they're a little bit pale so I will probably want to fix that. Um but you get this kind of like nice kind of I guess automated colored text in yeah it says the right thing at least and the right colors it does yeah. Um and the reason that works is because in inside the theme I have plot.subtitle equals element textbox simple which is from ggtext and it basically says apply any html formatting or markdown formatting to the text.

What I think might help is if I put those names in bold text just for some of those lighter colors right. So you can use like it's something that I think is a little bit weird to combine markdown formatting with html in the one line um but it works. So you can stick uh the names in bold text and in markdown and it just makes it a little bit easier to read.

Um and then you can go ahead and sort of start adding things uh like titles and captions um and whatnot. Um the other thing I want to do is move this down a little bit so in annotate um you have like hjust and vjust to move up and down um I can never remember which way around they are. This is where you type it and figure it out actually. I love teaching hjust and vjust but whenever I'm in some sort of like other coordinate system who knows yeah. Um so we just moved it down a little bit so if I go a little bit further okay um. It is supposed to be between like zero one but putting values outside of that does work. Um so putting two does shift it down a bit. Um and I think that is the benefit of coord radial is that like moving stuff up and down actually moves it up and down if you use coord polar moving it up and down like sends it like left and right diagonal yeah which is really confusing um.

Relative sizing, captions, and saving

So we kind of got this chart here um I think what I kind of want to do is because it's it's not super kind of explanatory would we make it um interactive. Um so the nice thing so if we put like the city interactive so switch geom col to geom col interactive and let's say the tooltip is just the city. Initially it's not going to do anything because with the giraffe package slash functions you have to put it in here.

Okay that's not working for some reason okay let's ignore that for a second um. What else do I want to do with this okay so the other thing I wanted to check was like does this actually work in terms of being parameterized so if I switch it to like a different city and just rerun all of my codes does this work. And like it does like you can see that it's the colors are still like the right way around they're still in the same kind of order nothing's changing but you're getting this updated chart um which is quite nice.

So yeah I think like we are starting to run out of time a little bit so I will yeah but look how far you've come in I know less than an hour this is amazing.

Um the last couple of things I'll show you is like um I guess like using like title um so title I might just do something like twinned cities like around the world. Um and again I've kind of like pre-populated that plot.title thing um so it actually uses a different font and like makes it bigger. Um one of the nice things about this is like relative sizing um so the idea with using size equals rel is that it's saying that the title is 1.5 times the base size which means that if you want to resize your chart you don't have to go through and like change the font size of every single element.

Um one of the nice things about this is like relative sizing um so the idea with using size equals rel is that it's saying that the title is 1.5 times the base size which means that if you want to resize your chart you don't have to go through and like change the font size of every single element.

Um so I think this is like quite small um and my base size and my uh theme is 12 so if I made it a little bit bigger okay like everything kind of scales up which is quite nice.

And the other thing I do have is um some functions um so these both I have two functions here one is like social caption and one is source caption which I appreciate are badly named because they do quite different things. Um so these are both currently in a personal R package but what these do is basically um sort of define the text I usually use in the source. Um so if I run this what it does is similar to that kind of um trick we did with the color text as it creates all of this um like horrendous looking html code um to stick some icons and like attribution in it. And then this um is where I would put like combine the attribution for the source of data with the attribution for the chart. And then if I do in here uh caption equals cap okay it adds like this at the bottom automatically for me um so I don't have to kind of think about that every week I sort of figure that problem out once um and then never do it again.

Yeah and that looks kind of okay and you can like adjust the margins um a little bit yeah. Um then yeah core fixed is really useful it's really nice for like heat maps when you're trying to make yeah not as helpful with this situation.

But we did have a canvas question um Daniel Chen asked um so you can save a plot with the canvas call and it doesn't mess up the ggplot object. Yes you can um that's a very good question um so one I guess there's two ways of thinking about this is one you can you can either like get rid of your kind of canvas call and like it looks horrible but then you can do ggsave as normal and it will look nice. Um the quicker way of doing it is actually the um the ggview package has two functions one is canvas the other one is save underscore ggplot. So I this is what actually converted me to being like a forward assignment um operator. Um so I usually have this commented out at the end of my templates um but this sort of says like assign that to the object p and then you can save underscore ggplot at the object p and that will save it um in the right place. Um and it's quite easy to just comment that little bit out if you decide to make some other tweaks to it.

Um but I can show you what that looks like oh my gosh look at all the nice little like social icons down there this is yeah adorable. So I did there is a blog post I wrote on like how to make your own and I think someone um converted that into an r package and I uh I cannot remember the name of it but it's something like gg like gg annotate or like caption I will look it up and I will find it. Um but okay we'll put it in the discord yeah once we find it like when you run um save ggplot like that's that's what you get um so it looks you know identical to this version here which is why I use it. What you're getting yeah know what you're getting you don't spend ages like styling it and making it pretty and then have to like do it all again when you decide what size your chart is.

What you're getting yeah know what you're getting you don't spend ages like styling it and making it pretty and then have to like do it all again when you decide what size your chart is.

Um so that's kind of like a really nice thing with that but yeah so I think yeah so I think like that's we've done like what like 15 minutes and that's kind of where we are I think. Amazing look how far the like helpful tools and you know I hit this problem once I'm gonna solve it once and have it solved for me and in the future look how far that can get you that's amazing. Yeah so like it really saves time doing those like repetitive tasks that you do over and over like you just save so much time if you don't have to kind of keep doing them.

Wrapping up

One of the um let me see one of the things I wanted to point out is I said oh it took me this long to realize that Nicola is like um in our brand was Nicola Reni brand oh yeah. And then someone mentioned that it's also why cowplot is called that because c-o-w was Claus O. Wilke. Oh nice blew my mind what a what a great piece of lore.

Um thank you so much for for sharing everything and all of your wisdom you have some groupies here if people would like to keep up with you and see your posts what is the best way for them to do that. Um I'm on blue sky and pink tin and mastodon or my website which has been there all of the other things yes. And like if you see the strip at the bottom of her her plot there you can see all of the places to find her.

And you I happen to know submitted this tidy Tuesday data you regularly curate so thank you so much for the kind of like service that you do to our community and helping curate. Thank you I won't fully take credit for this week's curation it was actually a colleague of mine that I worked with curated the data I just did the PR but the easy way at the end yeah the important part of getting it across the finish line.

Everybody I would really love to encourage you to curate a data set we have an entire episode that we did with John Harman of tidy Tuesday and if you are curious about how to put data together how to curate it and submit it I really really highly encourage you to do that it's such a fun process and then your data gets to be used by people all over the globe as they are learning and practicing their database um which is a really really cool feeling. I have submitted once and I got to see people make plots and I was just like mind blown.

Other than that Nicola thank you so much for joining us this was super fun I hope you had a good time. You're welcome I had a great time. Awesome everybody big round of applause for Nicola. Big round of applause for Nick who joined us as our special co-host today. Isabella we really really miss you we hope you're having a wonderful time at PyCon and we'll see you all um on Thursday for coming to the data science hangout or next Tuesday at the lab. Bye everybody I'll see you on discord.