R & Python playing nice, in Production (Claudia Penaloza, Continental Tires) | posit::conf(2025)
R & Python playing nice, in production Speaker(s): Claudia Penaloza Abstract: Hi, I’m Claudia Peñaloza, a Data Scientist at Continental Tires, where going data-driven can be an adventure. What started as a proof of concept a few years ago, evolved into Conti’s first-ever Predictive Machine Learning Model for R&D! A baby, a wedding, two lateral moves, and three hires later, our team had also evolved… from mostly R to mostly Python developers. Rewriting 1000+ commits? No thanks. Instead, we got R and Python to play nice. With Renv, Poetry, and Docker, we keep things reproducible, portable, and deployable on various ML-Ops platforms. The takeaway? With the right tools, teams can mix and match languages, leveraging the best in each, and still build solid, scalable solutions. posit::conf(2025) Subscribe to posit::conf updates: https://posit.co/about/subscription-management/
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hi, everybody. Thanks for having me and thanks for sticking around this long. There's one unfortunate thing that I just realized happens when you're the last speaker in a session of people talking about the same thing. Everybody has already spoiled my talk. So if you learn anything new, it probably isn't going to be about the R & Python playing nice part. Anyway.
So in the next 15 to 18 minutes, I hope, I will give you guys an introduction and a project overview. I will talk about how life tends to interfere with even the best plans and what strategies we use to get over those challenges that life threw at us in the middle of our project.
About Continental Tires
So I work for Continental Tire, or Conti as we like to call it. It is a big company. It's been around for about 150 years, builds all kinds of different tires, and I have a soft spot for Conti, specifically because of that third tire in row there, because those are the bicycle tires and I am a cyclist, as you can see here in this family photo of me with two of my favorite bikes. But no, we can't talk about bikes all day. We have to talk about tires.
So tires are surprisingly complex. An average passenger car tire can have up to 200 components, and those are built to certain specifications. If you look over here, aside from the rim size and such, we have the load index and the speed symbol for that tire. These specifications we actually have to test for to make sure that our tires are performing as we expect them to be.
So in Conti, we have been testing tires, well, obviously making tires for 150 years, right? We've been testing for a long time, too. We have a lot of data on previous tire builds and tests, and we wanted to combine the knowledge that we already have of tire testing and tire build to be able to predict the performance of tires before they are actually made.
The project and team changes
So we worked on this POC for about 18 months when it was approved for industrialization, and at about that same time, my project lead and peer, our developer, was also getting ready to deliver something. Her human baby. And because this is a German company, I don't know if I skipped that part, but anyways, she took parental leave for two years. Yeah, I know.
So yes, some parts of the world actually put life first. Anyway, at that point, the team changed, and we had a language flip in the developers. We went from a three-quarters R to Python to a, you know, Python to R, mostly Python developers.
We must rewrite this all in Python. This was heard in some of those early meetings with that new team. Actually, somebody who was not going to contribute any developer time was the one who said this. But at this point, and here I will digress with Blake, we had already put about 1,000 commits into the repository. We had the machine learning model already deployed to our platform, and the only developer really available to do that would have been me. So it was a non-starter.
Rules of engagement
The developers really had to lay down some rules of engagement. We had to agree how it was we were going to get this off the ground. So yes, the guys ahead of me spoiled this. We had to agree on data exchange formats, right? First thing in the pipeline, we wanted to make sure that any one script could take outputs from any other script. So we agreed to have our intermediate files would be parquet files. At the end of the pipeline or beginning of the pipeline, we needed to make sure that the rest of the company could access our results or that data from the rest of the company could come into the pipeline. So in that case, we had post-press SQL databases.
Another thing that happened in our daily development is that we agreed that we were going to do comfort zone coding. So no one developer needed to develop in something they weren't comfortable in. Everybody, like, if I wanted to code in R, I could code in R. The other guys could code in Python. No problem. No hard feelings. But we did decide that for code review, we were going to do cross-language reviews because we wanted to enforce four-eyes principles. We also wanted to make sure that everybody had a basic understanding of what the new contributions or refactoring to the code was going to be.
But when you review in a language that you're not very comfortable in, then, you know, you can't really make the code snazzier or super fancy, right? We had to adhere to core principles. So like really the basics, talking about, you know, can I understand this code because the documentation is sufficient? So making sure people were documenting their code. Making sure they were naming things in a way that, you know, future us or future some other team who is maintaining this will be able to understand what these objects or why they are called this or that. And, again, also trying to maintain, you know, function-based coding also.
Deployment and environment management
So after we got that settled, so our day-to-day, then we also have to think about deployment. When you deploy something, you have to make sure that your code will run reliably many times. So every time it runs, it has to run well. So we really had to get beyond it works on my machine. Thank you, Michael. I don't know where you are. So, yeah, we really needed to get beyond this mentality. Because this code is not going to run on your machine, right? It's going to be on the cloud. It's going to be on some, you know, platform, blah, blah, blah. So how do you take care of making your code run reliably? You need to manage your environments, right? When I submitted this talk, so for R, we were using renv for package management. And when I submitted this talk for Python, we were using Poetry. And soon after, UV came out and I jumped on that bandwagon and it has been beautiful. And here I disagree with Jeroen. Package management in Python is extremely easy with UV.
I'm going to also make a little aside here as to how things are different once UV came into the picture. So renv is very similar to Poetry, right? They both create lock files, which then Docker uses to create a Docker image. And those Docker images are those little containers in which we ship our R installation, our different packages, all of the dependencies, our code. Everything just lives inside of those little packages.
But when we move over to UV, UV happens to still use a base Docker image, but it installs all packages and dependencies on runtime extremely fast. And if anybody caught that— the talk about Rust and R this morning, it's because it's written in Rust. And, like, it completely blows my mind how fast this runs. What this means for me on a daily basis is that I do not need to rebuild an image every time somebody adds something. Because every time you use a new package, I would need to rebuild the image because, remember, this is running somewhere else. It's not running on my machine. So I need to rebuild the image. That can take really long. It's annoying. With UV, I do not need to do that anymore. I have heard that there is an RV in the works. And I have not had time to use it yet. It's actually already available. So I would love to be able to tell you that it works. I haven't used it yet.
Package management in Python is extremely easy with UV.
Containerization and orchestration
Okay. The next step in this process is containerization, which I've already spoiled a little bit. We need to use these Docker images to be able to contain our code in isolated environments. And we created one for each language. So all of the Python code runs in a Docker image. Okay. Scratch that. We have— UV using this, right? So we have a Docker image with a base Python Docker image. And then we have a separate Docker image for our R code. And once this all comes together, the orchestration is actually deployed on a language-agnostic MLOps platform. It doesn't care what you're running on it. All of the different steps in the pipeline are configured. How the different steps interact with each other, how— you know, what kind of resources you need, which outputs go where, which inputs get taken and where, that is all configured on this MLOps platform.
So our final infrastructure looks something like this. This is a diagram of what the machine learning pipeline looks like. We have a lot of AWS stuff in here. ValoHi is the MLOps platform that we use, which is based on Kubernetes. We have the EC2 instances, which are the orange squares, and the S3 buckets, which are the green ones. And the different— the combination of Python and R is something like this. So, you know, some of these EC2s run Python code. Some of them run R code. And in the middle, we're just saving the outputs and grabbing them from the next node with whatever is coming next. The whole pipeline looks something like this. As I mentioned, those two databases on either side. And a Tableau dashboard at the end, which the tire developers are using.
Results and takeaways
So we were finally able to get to a deployed multilingual ML model that is nowadays used by over 100 developers a day. And we can give them overnight predictions instead of them having to wait about four months for a tire test. So how do you make R and Python play nice in production? Or at least how did we do it? We standardized the data exchange formats. We took a pragmatic approach to coding and code review. And here I would say anything that works for your team is what works. Containerization for consistent environments. And language agnostic orchestration. And yes, we can make R and Python work together and be productive. And we can leverage the benefits of both languages without having to compromise reliability.
And yes, we can make R and Python work together and be productive. And we can leverage the benefits of both languages without having to compromise reliability.
Q&A
Excellent. We have a question or two. Why different Docker images for R and Python?
Yeah. I mean, there's really no need to load the R. I mean, I don't want to dis R. But the Docker image was about twice the size for R. It also had, like, we had everything that happened in the ML pipeline in the R Docker. We could have probably cut that up into ETL and maybe, you know, preprocessing and stuff like that. And then, like, the whole tidyverse gets loaded in one. But just having to load the whole tidyverse and tidy models and it gets pretty fat and heavy. So separate the Python from the R, and that makes both Dockers a little lighter and a little faster.
Okay. Are predictions on individual tires?
Yes. Yes, yes. And I'm going to elaborate a little bit. Right now we have a batch prediction for any tire that any developer has put together. We want to get to the point where they have an instantaneous prediction. Like, it's not a problem of our prediction speed. It's more of a platform inside the company problem that we haven't been able to give them an instant prediction.
Okay. All right. Let's thank Claudia and all the other speakers.
