What we're doing to make Quarto fast(er) (Carlos Scheidegger, Posit) | posit::conf(2025)
What we're doing to make Quarto fast(er) Speaker(s): Carlos Scheidegger Abstract: Quarto is a powerful system, but its performance leaves much to be desired. In this talk, I'll go through the things that make Quarto slow, and I will describe the journey I'm taking in 2025 to fix the issues. This is going to be a deeper technical talk on performance analysis, profiling, and will include discussing the custom tooling we've had to build to measure performance in a system as complex as Quarto. Quarto markdown repo: https://github.com/rstudio/rstudio-conf/blob/main/2025/github.com/quarto-dev/quarto-markdown posit::conf(2025) Subscribe to posit::conf updates: https://posit.co/about/subscription-management/
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hey, everyone. I'm not beating that. I'm Carlos, technical lead in Quarto. I want to start by thanking the Quarto team and all of you. This is just an absolute honor and a pleasure to be here.
Why so slow? I promise we're working on it. So let me just give you a story. This was going to be a talk about how Quarto is slow and making it faster. I got five minutes. We're going to say why Quarto is slow and what we're doing about it.
In 2024, about a third of the time was spent in TypeScript, which is the stuff that we use to write Quarto, and two thirds of the time in Pandoc, which is the thing that we use to turn your markdown into HTML, PDF, and so on. We spent a ton of time last year and this year deep into the Pandoc internals with the help of a number of collaborators and of teams, we sort of cut the time in half. And so as of this year, our website, which we render every time we make a change, is now taking four minutes and 45 seconds to render 400 documents, which if you divide one number by the other, it feels good, but if you have to wait five minutes, it feels awful. And so it's like 30% faster than 1.6. Great, I'm still waiting five minutes.
What's making Quarto slow
What's going on? So let me tell you what we are trying to do about it. So I'm gonna focus on what's happening with Pandoc, which is the thing that takes your markdown and converts into all of that. Turns out it's about half of the total time right now, and what's really surprising, it took, this took like two months to be able to come up with this actual plot of sort of figuring out how to instrument all of this and like learning about Lua and C and like patching the interpreter to profile, all a bunch of stuff that we don't have time to talk about.
I want to focus on that number there, right, sort of just reading the input before we even get to the part where we get to sort of run the things we need to do and output. Turns out it was taking 20% of the total time of the website, which is shocking to me as well, it's just a lot. And maybe we can do something about that. So that's what I'm gonna show you what we're working on.
But of course, that's not what you're here for. You're here like, what? I actually needed to be super, super fast, right? And you know, I have this computer with like 20 little computers inside it that can all work in parallel. And you're not using that. Why are you not parallelizing? If I change one page of my website, sometimes I hit render and like render the whole thing and you're just staring at it and say, why, why did it do that?
So why can't we fix those things? It's mostly because it's really hard currently for Quarto to know the relationship between all your web pages. And if you change something, we might need to change the sidebar because the titles have changed. We might need to sort of recompile a lot of these things. And Quarto's trying to be conservative because it's hard to sort of figure all of this stuff out. It's hard to figure all this stuff out because operating on all these documents for us currently is sort of painful because currently if we need to like look into the markdown to get to the structure, we have to go through Pandoc. And while going through Pandoc, we now learned it's sort of slow, so we looked at this and said, can we do something about that?
Rewriting the front end in Rust
And the answer is like, what if we sort of took over that little bit part of Pandoc ourselves, we know what the markdown looks like, we can try to sort of take over it. And so this is really a story about what we've been working on for the last six months, me being me. We are sort of taking over the front end for Pandoc and sort of parsing this ourselves. If you heard AIR, if you heard the story about Ruff, if you heard all of this, this is the same story. We're just rewriting the entire front end in Rust.
Cool is that this is a standalone library or a binary, so we get to use it, but you also will get to use it from R, from Python, if you need to do it in your own tooling. It compiles to WebAssembly, if you know what that is, that's great. If you don't know what it is, suffice it to say, it means that it will work inside Positron, it will work inside RStudio, it will work inside Z and your favorite text editor, so like all of that fast stuff is gonna get, you're gonna get to use it as well.
Here's the cool part. Turns out Pandoc doesn't give great error messages, you just saw sort of people organizing over semicolons, you make a mistake, where is it, what goes wrong? We know that Quarto does great error messages with YAML, not with Markdown, so we're like, is this an opportunity here to do something cool about it?
Well, so this is my favorite part here, so this is coming hopefully next year, but look at these error messages. So, oh, the color is not great, I apologize, but here's the cool part about this, so you know, if you make a mistake now, we hope to be able to give you an error message that looks like this instead, right? So you forgot to put a space there, instead of giving you some bad output, like you're gonna get, hey, in this file that you generated, like, here's a callout. Actually, you need a space before here, so this is where you need to put it, please add a space there, right, so all of the errors are gonna show up like this, hopefully before you render even sort of on, um, Positron or RStudio.
So it's nice and it's also fast, it turns out Rust is rudely fast, so this is like a 100x improvement, which was a stupid number for me to hear about, and from there we can sort of unlock parallelism and incremental rendering.
So it's nice and it's also fast, it turns out Rust is rudely fast, so this is like a 100x improvement, which was a stupid number for me to hear about, and from there we can sort of unlock parallelism and incremental rendering, so hopefully we'll end that in 1.9 in early 2026. If you want to keep an eye on it, that's the repository. Thank you very much.
