Posit Open Source
Resources

CI/CD Pipelines - Oh, the Places You'll Go! - posit::conf(2023)

Presented by Trevor Nederlof Data scientists are creating incredibly useful data products at an accelerating rate. These products are consumed by others who expect them to be accurate reliable and timely, often promises unfulfilled. In this talk, we will explore how to use common CI/CD pipeline tools already within reach of attendees to automatically test and deploy their apps, APIs, and reports. Presented at Posit Conference, between Sept 19-20 2023, Learn more at posit.co/conference. -------------------------- Talk Track: Lightning talks. Session Code: TALK-1166

Dec 15, 2023
5 min
RStudio Data Science Machine Learning Stats Tidyverse Data Visualization Data Viz Ggplot Technology Coding Connect Shiny Rmarkdown Package Manager CRAN Interoperability Serious Data Science Dplyr Forcats Ggplot2 Tibble Readr Stringr Tidyr Purrr Github Data Wrangling Tidy Data Odbc Rayshader Plumber Blogdown Gt Lazy Evaluation Tidymodels Statistics Debugging Programming Education Rstats Open Source Oss Quarto Posit Python ML

image: thumbnail.jpg

Transcript#

This transcript was generated automatically and may contain errors.

I'm Trevor Nutterwalf, a solution engineer at Posit. And back in 2016, I failed my coworker, Sam.

So imagine this, Sam, late from his commute, has 10 minutes to spare before an important executive meeting. He comes racing in the office, fires up a shiny app to grab the latest report, and this happens. Oh, no, undefined column. Sam quickly has to grab the report from yesterday and explain it well. The meeting did not go well. I failed my coworker.

What went wrong in 2016

So Friday, I push a tiny fix to my app. I give it a few quick tests manually, and it's the end of the day, and so I forget to deploy it, but I figure I'll do it early on Monday. Monday rolls around. I deploy the app, thinking it's working, and then the fateful Tuesday happens. Sam has trouble getting the report, and it really hurts his performance.

An alternative reality with CI/CD

But the good news is that was back in 2016. Let's look at an alternative reality if this happened in 2023 with better tooling.

So now I make my tiny fix, but this time I push it up to my GitHub repo, and my CI/CD pipeline runs, does a test, and flags the error. The shiny app doesn't look right. That's great. I can go back, and I can make the fix on Monday. I get in a little early because I know the error occurred. I saw it on my email. I make the changes. I push it up to GitHub. My CI/CD pipeline runs. Tests succeed, and great. It deploys the new version of my app to my server.

Now Sam runs in, 10 minutes to spare, goes to the app, downloads the report. It comes out really fast. He can drink his coffee. He gets his caffeine hit. He goes to his meeting. He gets a huge promotion. He becomes my boss. He's the CEO of the company, and he buys a sports team someday, a wildly successful career. What a great alternative reality. We should all live in this world.

He's the CEO of the company, and he buys a sports team someday, a wildly successful career.

Setting up the pipeline

So first, I set up a CI/CD pipeline. I use GitHub Actions, but there's other tools I'll talk about at the end. There's two main pieces, continuous integration, running the tests. I used a package ShinyTest2, which allows you to record your screen. It's great and record how your app behaves. And then continuous delivery. This deploys my app to Connect only if the CI steps are successful. So only at the end of the pipeline.

It's a great time to be setting up CI/CD pipelines. And this is because in GitHub Actions, there's a lot of community pre-built steps. I've left some links here. RLib has a number of them useful for R users, and RStudio has some as well. And the purpose of this talk is not to go through code, but I did want to show just how easy it is to set up your own pipeline. You can compose different steps, kind of like packages, into your pipeline.

So here, I set up R, I set up my renv environment, I run the test, and I publish to Connect. It's really this simple, and you can do it too. If there's one takeaway from this talk that I want you to have is to run out and set up your own pipeline. What do you have to lose?

If there's one takeaway from this talk that I want you to have is to run out and set up your own pipeline. What do you have to lose?

And there's a couple benefits. The main ones are deploying from Git repos. How nice is it that I can just commit to my repo and then trust my pipeline will do the rest? It'll automate the tests, it'll only deploy the content after testing. And the result of all this is my users will trust me. It's a good feeling to know that when you log on to an app, it's going to be successful. And that grows the user base within your company.

And there's a lot of other tools. I talked about GitHub Actions, but if you're using any of these other ones, contact your DevOps team or others on your team that have access and get started. They all are fairly easy to use, and your team may already have some actions in the pipeline that you don't even know about.

I'll leave you with a few resources. You're always welcome to contact me and reach out with any questions. I'm happy to help. Get out there and make some awesome pipelines.