Ben Matheson | How Anchorage Built Alaska’s Vaccine Finder with R | Posit (2022)
In January 2021, Alaska residents seeking a COVID-19 vaccine appointment faced a convoluted maze of websites. The software was made for providers—not for residents. The Anchorage Innovation Team built a fast, and mobile vaccine finder website for Alaska using R. What started as a web scraping prototype launched statewide one week later and ultimately connected tens of thousands of Alaskans to a vaccine. This talk will cover how we used R to build Alaska's vaccine finder. Including: • Scraping and http packages (rvest & httr) • Using Heroku and S3 to run R jobs 24/7 • Creating a flexible data service with R Session: R be nimble, R be quick, R help me plan my vaccine stick: Rapidly responding to world events with R
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
In January of 2021, Alaska was on the cusp of the most highly anticipated vaccine launch in 50 years, the COVID vaccine. So as the state began to roll out shots to senior citizens aged 65 and above, the hottest ticket in town was a COVID-19 vaccine appointment. So when a batch of appointments would come online, those would be snatched up within hours or sometimes even minutes. And back in those days, everybody needed an online appointment. You couldn't just walk in and get your COVID-19 vaccine.
So it's a really exciting time, but we found out pretty quickly that the website the state was using to get people signed up was not working well for senior citizens. It was hiding the appointments in several pages of results. It was made for healthcare professionals working on desktop computers, but it wasn't working for people on the phones. The maps were distorted. It's showing the Sacramento County Jail instead of the Anchorage Health Department here.
And so we started to hear from senior citizens that they were struggling, that they were not making it through the appointment registration process. And so at this really critical juncture, the senior citizens are starting to contact us and saying that some are at the cusp of actually giving up. And at this really important moment of the pandemic, senior citizens are frustrated, the state is frustrated, and I'm frustrated, especially when I learned that there's no changes and no improvements that could come to the website anytime soon. But I knew that there must be a way to improve this website experience, and I knew I had to at least try.
So my name is Ben Matheson. I work for the Anchorage Innovation Team in Anchorage, Alaska. We're a small team inside city government that uses data, human-centered design, and technology to solve problems. So we try to improve the lives of residents, solve problems with city departments, and save money.
So today I'm going to tell you about how we built a vaccine finder for the state of Alaska in one week using R. And I hope to show you how R is a great tool for rapid prototyping and rapid building of products. And after this, I hope that you can better appreciate how you have the tools and abilities to really make some cool custom rapid products.
Validating the problem
And so before we started building anything, we wanted to validate if there was a problem and find out exactly what this problem was. So we got on the phone with senior citizens as they were trying to sign up for their vaccines, and we found out a few things. One is that they are struggling through a complex website. We learned that they're struggling with some pretty complicated health questions that show up in these vaccine registration systems, and also learning that the adult children sometimes of seniors are the ones who are actually signing up for appointments.
And so we tried to narrow the problem a little bit to, you know, how can we help Anchorage senior citizens who want their vaccine really quickly find that appointment so that they can get their vaccine?
Building the MVP
And so before we started building anything big, we wanted to do a no-budget build, an MVP, a minimum viable product. So in this case, what is the smallest, fastest, quickest thing that we can make that proves that there is value, get it into the hands of people, and see whether we're helping the problem or if we need to go in a totally different direction?
And so this was a literally Friday night project. It's Friday night in early January, and I picked up the tools that I use when I want to make something fast and easy and maybe not easy, but I picked up dplyr. I use Stringer and Lubridate, and we're trying to capture vaccine appointments as they come online and as they disappear, and we learned pretty quickly that there's no API coming. There's no programmatic access to vaccine appointments in Anchorage. So it's time to scrape.
So we used rvest to scrape the vaccine web pages. And anybody who's ever used a web scraper knows that it's like trying to create order from chaos. You know, you find a div that appears to have the information that you're looking for. You know, it's border gray 200 is the div that we want here. We go grab that div and extract the text from it, and from that text, we try to turn that into structured data that serves our needs. So rip it apart, mutate, modify, put it back together, and make structured data out of this text.
So we're scraping. We're able to capture that data, but then it's time for a really quick front end of how do you discover vaccine appointments? And because this website was a reaction to an overly complicated website, we wanted to go with something super simple, super fast, super lightweight. And so I went with an HTML table.
So on the left is the prototype that came together after a few hours on a Friday night. It's a web scraping service that scrapes every 60 seconds to try to find those appointments as they come up, as they disappear, and we show those in a really simple format. No filtering, no menus, no maps, no anything. It's how do you really quickly find those actual vaccine appointments?
And then it's time to test. So we've got this very prototype website, and we send it to people who are working on the Vaccine Call Center. It's a team of about 40 people who spend all day on the phone with senior citizens trying to help them sign up for appointments. And they're able to give us some really good feedback. They're able to see if it's helping. And it did help. They're able to kind of use it as a power tool. This was the Vaccine Call Center's power tool that we tried out.
And then something else happened, in that this website started to be passed around between people. And it's not a real website. It's not a real product. But over the course of a weekend, all of a sudden, there's hundreds of people using this website and signing up for appointments. And we're hearing back, and people are having a good experience. So we know that there's something there.
Launching statewide
But it turned into an unexpected launch. We've got this product, this prototype. But the state liked it. They wanted to use it. So we developed a really quick partnership with the state health departments, the Anchorage Health Department, and the Emergency Operations Center.
And nobody from Alaska can do a presentation without a map of Alaska. But I just want to clarify, I work for the city of Anchorage. So we're the biggest city in Alaska, about 40% of the state's population. But what the state wants to do is to use this scraping-based vaccine finder for the entire state. So they want to use it in Fairbanks and Juneau and Ketchikan and Kodiak. And I'd say that's awesome, that's great. But we have to scramble here.
And so over the course of the week, we cleaned up the scraper. We made it real by buying a domain. And about one week later, we launched. So we had an Alaska statewide vaccine finder tool. It's what I showed before, but basically cleaned up. It's appointments that you can find by location, address, vaccine type. It's text above, you know, on the phone that works well. And that's it. That's what we generated, and that's what we started using.
And so over the course of January and February, we used this for things like signing up people for these big mass vaccination clinics at a basketball arena. Or when we launched drive-through clinics at hockey arenas, we signed people up through this website. So that's a lot of fun. And we got some good feedback. People were able to use this to connect to their appointments. And this tool was doing what we hoped it was doing.
Adapting to the unexpected
But it wasn't always straightforward, it wasn't always easy, and things definitely don't always go according to plan. So on March 9th, 2021, we got word that the governor was going to make a big announcement. Alaska was going to be the first state in the country to go fully open for everybody. Everybody age 16 and above would be able to get their vaccine appointments.
So at 4.44 p.m., I had to send this message to my boss. The website was down, we had some weird data come across the scraper, and at the moment that we were expecting the biggest surge in traffic, maybe the highest amount of interest over the course of the entire rollout, I have to deal with this. But I was able to filter it out, keep it online, and I was really happy that we could put this banner up saying everybody age 16 and older can now get their vaccine in Alaska. So we stayed online throughout that pretty busy day.
And over the course of this project, we didn't have the most stable website, we didn't have the fanciest website, we didn't have the fastest website, but our measure of success was how fast can we adapt to everything, adapt to new eligibility, adapt to new data, adapt to new pharmacies, adapt to new requirements, and that's exactly what we had to do.
fast can we adapt to everything, adapt to new eligibility, adapt to new data, adapt to new pharmacies, adapt to new requirements, and that's exactly what we had to do.
This vaccine rollout was one where Alaska was first, and we couldn't copy anybody else. We couldn't copy California or New York or Washington, but we kind of had to make it up as we go. If we could copy, we would, but we just couldn't.
In some projects, the timelines aren't what you expect. In early January, we heard a really optimistic timeline that vaccines would be open by the summer solstice. So approximately a six-month rollout of those vaccines before they open up. But here it was March 9th, and it's the Iditarod Trail sled dog race, and we've got vaccines rolling out to everybody. So this whole project moved even faster than we ever really had expected.
And we rode those waves throughout January and February as we opened up to younger people, as we opened up to new eligibility. And we got some, for Alaska, some major traffic. We'd see 20,000, 40,000 page views on a given day. And for this scraping-based website that has no backup and no team besides myself and my boss, we got a million page views. We connected the thousands of Alaskans to their appointments, and it served as the Alaska statewide vaccine finder tool for many months.
And on a personal note, I got my very first piece of fan mail for an R project. And I definitely sent this to my boss as soon as I got that email.
Making it more stable
But as we moved out of those really crazy days of the early pandemic, we wanted to turn this into something more stable. I'd spent enough time trying to fix a scraper at 9 p.m. or 6 a.m., and one of the big changes was that we started with one major data source, and that's scraping this system called PrepMod. But all of a sudden, we were bringing in all sorts of other data. We've got pharmacy APIs. We've got some open-source project that brings in pharmacy chains. Vaccines.gov came on board, and we accessed some of that CDC data. And then all of a sudden, this simple project turns into kind of a complex project with six different opportunities for things to break.
And what I was definitely overdue to do was to turn all these into modules so that when something breaks, you can turn it off. When something needs to be troubleshooted, you can target that in its own individual file and its own individual function. And it made it from a brittle website into a less brittle website.
I also tried to bring in more error handling. Errors would break the website in the early days for sure. So I brought in a bunch more try-catch statements so that you can run a statement, and if it successfully executes, you get your data back. But if and when that error comes, we can still return some valid data. We can still keep the rest of the sources online, and we don't break the system just because those errors did come.
For automation, we really wanted to track appointments 24-7, because they were really going fast in those early days, and so we wanted to scrape every single minute to see how close we could keep up with vaccine appointments. And the way I did this was with a Heroku R buildpack and a Heroku cron job, where I basically ran one script every 10 minutes, and then in those 10 minutes, I scraped several times. So seven or 10 times every 10 minutes. We scraped and we scraped and we scraped and we scraped, and about 5 million HTML pages I think we scraped before we finally moved off of that scraper. So that prototype turned into something that got used quite a bit.
We also had to adapt to the complexity that came up throughout the process of this. If you may remember, there's a lot of phases. There's phase 1b, tier 1, phase 1a, tier 3, and we got a lot of requests to have information like this on the website, and we don't like too much complexity, and we're pretty militant about having simple information as much as possible. So the way that we did this on the front end was we kept the appointments above the fold, as they say, so appointments up high, but we radically simplified the eligibility information and kind of had it slightly hidden. But we kept the scope of this as narrow as we can to here's where you find your appointments.
And then more complexity came in in that all of a sudden we've got a lot of vaccine appointments and supplies at a certain point, and it's turning into a big list that needs some management. And because we didn't want to add complicated menus and we wanted this to work really well on phones, what we did was just little HTML buttons that allows you to search this table for the city or region, and there's a lot of string searching going on in this. So that was our strategy for dealing with that complexity.
Lessons learned
So what have I learned throughout this experience? One is that there's a really subtle power to creating an MVP, to creating a demo. So if we had asked in advance permission to do this website, we would have encountered a lot of reasons to say no. It's like what's your budget, what's the plan, what's your backup plan, who's the staff in charge of this? But when you put something in the hands of people where they can interact with it and break through all those barriers, there's really a lot of magic to it. And these prototypes, in our case, they're not pretty. They're text on a phone, but sometimes a prototype, just a proof of concept, can sometimes blow people's minds just enough to break through and make progress on projects.
There's also some secret value in building your own dataset. And in this case, we were kind of the only people scraping and aggregating appointments across the state of Alaska. And because we wanted to track how many appointments there were, we had the tools to do that, and I displayed how many appointments were available at any given moment throughout the day. And this was basically out of curiosity because we wanted to see how fast things were going. But at some point, we learned that the state health department was actually watching this number and actually using that number to make decisions on when it's time to open up eligibility or when it's time to make some changes. So when you build that dataset, sometimes you have the ability to find value and find unexpected value in places that you didn't intend.
So when you build that dataset, sometimes you have the ability to find value and find unexpected value in places that you didn't intend.
So R is a great prototyping tool, especially with all of you who love R and have spent some time with it. This project was a series of data frames that basically would be created, updated, modified, and then destroyed every single minute is what we would do. And because I'm comfortable manipulating data frames, that's what I did. And it's ultimately this data frame that took a lot of work, but that data frame turns into the product. It's the world's simplest database that is JSON or CSV, but that turns into the actual product.
So if you have that ability to work fast in R, you're really set up well to create prototypes. And in some projects, if this project had a different timeline, and we knew we had months to work on this, I may have used something more traditional like Node.js or Python or something more standard to build out a web service and a website and maybe use some React or something, but this is a project where the timeline was now, and it's a timeline of the prototype is now the product. So in those projects where speed and flexibility matter most, I think R is absolutely your superpower.
So if you've put in the reps and have that ability to work fast, you're in a really great place to do that. If you haven't put in the reps, you're doing it right now. And I think that you all should appreciate that you have the skills, the abilities, and absolutely the tools to really quickly make custom software or custom analyses or models. And your ability to do that fast, I think, can sometimes be a huge advantage in projects.
So if a computer is like a bicycle for the mind, I have to think of R as like the electric cargo bike. It's something where you can help a lot of people. You can do absolutely anything with it, and you can go fast and really have a lot of fun. So thank you very much.
