Quarto: Elevating R Markdown for Advanced Publishing | Christophe Dervieux
In the dynamic landscape of data analysis and scientific publishing, R Markdown has been pivotal for the R community, allowing users to seamlessly blend code, narrative and results in a cohesive narrative. Now, Quarto emerges as a powerful tool that builds on years of experience but also goes beyond R Markdown, providing more flexibility and power in scientific communication. This talk aims to present Quarto as the new alternative for scientific publishing. We will delve into how Quarto enhances the user experience for R enthusiasts, maintaining the syntax familiarity of R Markdown while introducing innovative and improved functionalities across multiple formats, similar to R Markdown ones. Why switch to Quarto from R Markdown? In which cases? How does Quarto integrate with existing workflows? Hopefully everyone will feel inspired to try out Quarto! https://quarto.org/docs/get-started/ Timestamps: 0:00 Introduction 0:41 Quarto is an open-source, scientific and technical publishing system 1:22 Computational documents and scientific markdown made easy for single source publishing 3:08 How to use Quarto 4:24 Quarto works with VS Code, Positron, Jupyter, & RStudio 5:22 Quarto's multi-language workflow 7:21 Quarto syntax 8:40 Quarto formats (html, pdf, docx, typst, beamer, pptx, revealjs, etc.) 12:19 HTML Theming 14:10 Typst CSS for nice table output in PDF 16:24 Publishing (Quarto Pub, GitHub Pages, Posit Connect, Posit Cloud, Netlify, Confluence, Hugging Face, etc.) 17:36 Shortcodes 19:10 Quarto Extensions 19:49 Quarto Projects 22:53 Project configuration examples for a website and a book 23:42 Resources to get started!
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hi, I'm Christophe Dervieux. I'm a software engineer at Posit and I'm working in the Quarto team. I'm also working in the R ecosystem, in the R Markdown ecosystem, for those who may know me for that work. But today I'm here to talk about Quarto and how it will help you improve your R Markdown publishing. So this talk and this presentation is mainly aimed for R users who know R Markdown, I would say, and want to discover and know more about Quarto. But it's also like a generic introduction on what Quarto is and how Quarto can be useful even if you are not using R. So you can also watch this video for this purpose.
What is Quarto?
First, I wanted to start by a reminder of what is Quarto. And so Quarto, it's an open source scientific and technical publishing system. So just like R Markdown, you can wave together some narrative and some code to produce some formatted outputs. And really Quarto is the next generation of R Markdown.
And so why do we say that next generation? Because it is not replacing R Markdown in the sense like R Markdown is not going away, but Quarto is next generation as it unifies and improves the R ecosystem. But more importantly, it will extend it and also for people who don't use R Markdown. So why we need a new tool, this next generation tool? I would say the most thing is that we want computational document and scientific markdown to be easy to do for what we call single source publishing and also design from the start and multi-engine. And so it's an attempt to bring the 10 years of the experience behind the R Markdown ecosystem, everything that has been developed over the course of 10 years from the core packages to the broad ecosystem, to bring that to new tools, new way of doing it, like a redesign of those things, but also extend it to the most people.
And really computational document and scientific markdown is something important. So you have this graphic that we use. This is really all the graphics you find the source on the right side of the slide that aims to show what Pandoc, which is a universal converter, is helping to do by allowing to write Markdown to Word and LaTeX. And when you do scientific publishing, LaTeX can be really useful, but can be quite complex to use. Like it's a slow curve. Word is really easy to start with, but when you want to do like really complex editing and also inserting code, it's quite complicated. And Markdown is kind of a mix of both worlds where you can do like really easy writing. You can also have a good way to execute code into it, but then we need a scientific layer and then what you want to do, technical publishing through Markdown. And Quarto tries to bring that with scientific Markdown and computational document, and you will see a bit later how does Quarto do that.
How to use Quarto
Quarto is not tied to a specific language like our Markdown is. It's a command line interface. It means like you can just download it, install it, and you get everything that you need included into it. And you can use it to render plain text format. So, the main extension being like the QMD extension. Or also mix format like iPower and NB, Jupyter Notebook can be an input format for Quarto. And you can render those documents to static files like PDF, Word document, HTML, but also website, dynamic presentation, and even more that we'll see in the course of this presentation.
And so, a command line tool, once you install it in your pass, you'll find the Quarto binary, and then we are now at the one file version, which is the last one released. And the main two functions will be the render function to render your file, and then the preview function. That's render and preview the document while you are editing. But it's also a tool that contains some helpers that I won't dive into specifically, but you can see our doc and see what they do. But there is already an ecosystem around Quarto, which this command line allows to integrate with.
But also, Quarto integrates with other tools. And this is really important because like today, even like as software engineer or technical writer, we could use a command line often. We usually works in other tooling like IDEs. And Quarto will integrate with those tools. It will integrate with VS Code because Quarto has VS Code extensions. It will integrate in Jupyter because Quarto has a JupyterLab extension. So, if your main source of document is IPython and you like to work in Jupyter, you can use Quarto directly in this JupyterLab IDE. And obviously, it's also integrated with RStudio, which was the main IDE for the R Markdown ecosystem.
And it's not in that slide, but today it integrates also with Positron, which is a new IDE by Positron. And we choose the same Quarto extension as the VS Code I showed just before.
The Quarto workflow
So, that's the generic introduction on what is Quarto and how to use. And so, now let's see the workflow on how Quarto works. So, if you know the R Markdown ecosystem, this schematic will be very similar to what you may know. So, we have a Quarto source file, which is a QMD document. And Nitr is used as engine to render any R code in this document to produce an intermediate Markdown file. That pen doc, which is a universal converter, is used to produce output documents.
And if we look at another engine, like if you are working, for example, with Python as your main language, you could be using the Jupyter engine. And you see that we just change the Nitr block by the Jupyter block, meaning the workflow is the same. And this is where the part of multi-engine is important. Quarto can use different engine. And so, here, QMD goes through Jupyter for execution and then to Markdown, which goes to pen doc. And Quarto will do some processing at each part of this step, before, after, Nitr, and also heavily modify the behavior of pen doc for bringing the layer of scientific Markdown.
This also means that I said just before that IPAWNB could be an input, and the workflow will be the same. Just we skip the QMD to Jupyter execution, and we can take an executed Jupyter notebook as input, for example. And so, it's not in that slide, but since the last version, Quarto 1.5, the community has contributed a Julia engine. So, now, you could have QMD goes through Julia for executions, Native Julia for executions through Julia engine, and then to Markdown. And so, it's proved that this is what we want to aim as a tool, Quarto, which can be really multi-engine for the future and any engine that may come tomorrow.
QMD documents: syntax and formats
So, that's the main workflow and how that works. Let's see what is really a QMD document. This is for our user mainly. We use R Markdown. And this is a R Markdown document. We have some YAML header with some option, a format, which is passed at the output key, and then some cell option that are passed on the same line as a starter of the cells. And the Quarto document is not that different. We will have still a YAML header to define the options, but now the keys format, and all the cell option will now be inside the cell using the YAML syntax. This different way of setting ocean in cell is an important difference, but it's really aiming to bring better user experience, because if you are using this in an IDE, you will get validation and auto-completions. And it's also a cross-format way to do that, meaning it will be the same syntax. So, I gave an example with the R cell, which will be using the NetR engine, but if you are using the Jupyter engine, you will have a Python cell, and it will be the exact same syntax and the exact same options. Quarto will do the translation and use the right engine, but the syntax is the same, and this is key.
Quarto will do the translation and use the right engine, but the syntax is the same, and this is key.
So, here goes for the syntax. And once you have this file, you want to convert to a different format. And this is where Quarto unifies and extends, because Quarto comes with multiple fresh polished formats, which are built in the same tool. You don't have to install anything else, and we'll have a consistent syntax across the format and across the feature. And so, here's a quick table of the different formats available. There is a parallel with the R ecosystem for those who know those functions. But we'll find HTML, PDF, Docx as output format. With the newcomers, which is a typed output format. Types is a new tool to convert to PDF.
Beamer presentation, PowerPoint presentation, and HTML presentation through the revealjs framework. So, in Quarto, we have only one framework, which is revealjs to make the slide, but we brought some of the feature for what was existing in the R ecosystem in this single presentation format. We will have a feature also that works across all formats. We'll find cross-reference, and in the R ecosystem, it was mainly available if you were using BookDance format functions, which are HTML document, PDF document, Word document. Now, you can use cross-references any other format. It's not tied to another tooling you need to install. It will be the same for advanced layout, which is a way to use margins, to use, like, reference on over, like, things like that. It was available in the R ecosystem. Now, it unifies in Quarto itself.
And then we will find the more complex formats, which are usually project format, and Quarto is designed from the start to work well for projects. And we will find a website with blogs, which is a subset of website with some other feature. In the R Markdown ecosystem, it was blogged on, or distilled, or R Markdown website, like, you needed to choose your tool. Now, in Quarto, you have, like, website project format, and then you can use any feature you want in Quarto itself. We'll find the books format, which is HTML book will be a specific type of website, but then you can do EPUB book, PDF book, so single file format like that. And then we will have even interactivity with interactive documents using Shiny, which was possible in the R Markdown ecosystem. In Quarto, you can use Shiny for R, you can use Shiny for Python, but you can also use other framework, like ObservableJS, that is available natively in Quarto. And the newcomers is dashboards. We squash the dashboard now, which, like, brings what was available in Flex Dashboard, but in a redesigned way, and making the logic of making quickly a dashboard for results available for Python user also using Jupyter Notebook.
So, you see, it's quite a large panel of format and features. And this is really where the unifies and extend part I was talking about at the start makes sense. Like, Quarto was built on all those experience across the league of what is needed for technical publishing, what is needed, like, for scientific publishing, and bring them in a single tool available, like, to the most of the users. So, for the rest, I will focus on some specific features among those formats. Like, it's not possible, like, to detail and to go in detail into everything. So, it's just a focus on some of the new things and some, like, of interesting feature, but there is a lot to discover and a lot to go. You can go on the website and probably see, like, some of our past and future video on those topics.
HTML theming
So, let's talk first about the seeming. Quarto have really a new way to do HTML seeming, which is inspired by what was available in the Armageddon ecosystem. So, we call that Quarto seams. You can have Quarto seam files, which are based on the SCSS syntax from the SaaS tool, S-A-S-S. And it's available for all HTML outputs, either website, which are using Bootstrap, HTML documents using Bootstrap, but also revealjs, not using Bootstrap, but using a SaaS seam also on the inside. And so, those seam files have a specific syntax that you can see on the left part, where you can define some variables, you can define some rules, and you can do much more if you know how to use SCSS. But those are the basic things that you may need to know. And in this example, I'm, like, changing the font size for any H2 headers and changing the font weight for all my headings. And then adding custom variable, which is custom color that I want to use in all of my headers. And so, I'm setting that in the rules. And by providing that seam file in my configuration, so in my YAML header, as you see on the right, I will be able to modify my default seams, which here is litera. And so, you can guess that this litera seam is one of the default available seams, which is from Bootstrap. And we bundle, like, 20, 25 seams in there. So, there is really a nice way to not use a default seaming for all your presentations or all your production. You can have seaming. And we are also working on, like, improved way to do that for the coming versions. So, stay tuned on this seaming part.
Typst: a new PDF engine
The second focus I want to make is about types. So, if you don't know types, it's a typist. Pronunciation depends. Who says it? It's a new PDF rendering engine that is available in Quarto. So, the website for this tool is a types app. And we bundle types binary inside Quarto. So, if you know how PDF work with latex, you need to install a latex distribution for that to work. The great thing here, if you do format types in your document, you will have a PDF version. It doesn't go through latex. It goes through types. And it's the quickest way, really, to do PDF with already nice features. And one of these nice features is the following. It's a recent addition also in the latest Quarto 1.5. So, I really invite you to try that out. Is that in Quarto, we are parsing HTML tables. So, if you produce in a code cell, for example, HTML tables, it will be parsed to a table and output as a table in any of the formats. So, if you have an HTML table using the GT package and you want to produce a Docx document, it won't be ignored, even if it's HTML, because we parsed it. And you will have a table in the Docx document. Though, we won't be able to keep the style. Because GT will use HTML styling and it's really HTML code. And so, usually, when this is done, the style is different on the output. And it would happen for latex, for example. But we found a way to keep the CSS. And so, you can get the exact same styling in your HTML document and your PDF document if you are using style. And what you see on the right is the exact styling of the palette that is used in GT and that will be used in the HTML output format. You will have the same table in PDF in types. This is like a screenshot of the example I did in a cell. So, this is a really cool feature. And we will keep improving the types format because there is really interesting feature to bring and compatibility to have one source file and multiple output formats that match each other.
Publishing and Quarto-specific syntax
Another specific feature of Quarto is the publishing that is built in, integrated into the Quarto command. Usually, when you produce a document, you want to communicate on something so you can send it to someone, like send a file or something. But when you produce a HTML presentation, a website, or even a single document, you want to publish it and be able to send a link to someone. And this is easily done with a Quarto publish command that will render your target and then publish it to one of the available providers we have. And you can see the list. We have the Quarto Pub, which is a specific service designed for Quarto. So, it's mainly for HTML document, but it's also integrated with GitHub Pages, Netlify, that are like wide-use service, even ng-face spaces now in the latest version. And obviously, some of other Posit products, which are Posit Connect or Posit Cloud. And so, you can easily, with one command, publish to those environments and have your production online really quick. So, this is really interesting, like nothing else to install or to add or documentation to read.
And the last focus I want to make for this presentation is showing what is specific syntax. Like I told about like scientific Mardon, but also Quarto-specific syntax. And we added, like Quarto is using Pandoc. So, anything that you can do with Pandoc can be used with Quarto. Specifically, we are leveraging Pandoc's Mardon syntax, but we also are adding some features like short code, which is a Quarto-specific syntax that allows to quickly insert some content in a document and do, for example, different rendering depending on the output format or depending on inlines or blocks, things like that. So, this is a really simple example with a new placeholder short code that we added that is a really easy way to add a dummy image, which is quite useful if you are doing like reproducible example, if you are doing like teachings, or if you want quickly to like design or iterate something, you can use these specific short codes. And if you are producing a HTML document, it will like write the correct HTML code for you, generating the image. If you are using a PDF output, it will do the writing also. So, this is one of the built-in short codes. You will find the list on the website. And there will probably be more coming built-in in Quarto.
But it's also a way for me to introduce you to the Quarto extensions. I won't go into details in there. I'll just put the link in the slides. But short codes can be created by anyone wanting to extend Quarto. And short code is not the only thing you can extend. You can do custom formats, you can do custom projects, you can do Lua filters, which is a way to extend Pandog behavior. There is already some API and some way to extend what Quarto default can do if you want to do more, if you want to provide some other default to like your community, your user, your use case, things like that. So, Quarto extension, if you want to do more, go look to Quarto extensions. And on our website, there is already a gallery and a list of what the community has done since the start of Quarto. And there are really a lot of extensions today available.
Quarto projects
So, I talked mainly about document syntax. And I mentioned briefly project, that Quarto was designed for project. And this is, as R Markdown user, it will be a big evolution because R Markdown from the start was not like really well-designed for project, like Bookdone was, and then Blockdone, like this notion of project came up later with several configuration file. In Quarto, it's much more simpler. A Quarto project is a directory that contain a file, which is called Quarto.yaml, period. This is a Quarto project. If you have a file with a Quarto.yaml, then this is a project.
And why would you do that? Legitimate question. It's to organize content to gain some project feature and enhance outputs. So, by enhance outputs, I mean project types. If you want to do website, a book, a manuscript, which is a specific types, some blogs that I mentioned before, like you need to do a project. Those work like project because you organize several documents together. But you can also do a project with only revealjs presentation, for example, but all those presentations, like share the same themes, needs to behave the same, or you want to leverage some of the project feature for all the presentation, you can use the type default and just say, this is a project, Quarto behave like a project.
And what are those specific features? Quarto has shared metadata for project to subfolder level. So, you can do customization and configuration for the whole project, for part of your presentation, for part of your project, sorry, like subfolder for a single document. So, there is a way to like have really multiple types of output in the same project. It brings some feature for reproducibility with computation, caching, which comes with engine, but also a new thing called freezing, freeze options that allow you to have a project where computation won't be run anymore unless you ask for it. And it's a good way to ensure reproducibility over time for your output. Like obviously not reproducibility for your computation, but for the website you're building or things like that. Quarto has also a feature for like pre and post-render script when you are using project. If you need to do some things before rendering all website or after rendering all website. And also way to have distinct behavior using project profile.
So, just names, it's maybe like very vague today, all those, but it's just to show that what you need to understand and keep in mind for this, like the takeaway message for this Quarto project that it exists. You may need it. Go read the doc about like each feature I mentioned and see like maybe it can improve your experience with Quarto or any of your current like work using technical publication.
Go read the doc about like each feature I mentioned and see like maybe it can improve your experience with Quarto or any of your current like work using technical publication.
An example of configuration or two example, I would say one for the website and one for the books is just to show you how simple it is and what could be the difference. So, we have the type at first that I mentioned. And then if we, the website, we are giving website configuration. If this is a book, we are giving book configuration. So, there is like specificity on these, but after it's like common configuration. And so, the format you see at the end of those means that every QMD document or ARPA 1NB document in my project will be rendered using this HTML theme with this style and like activating the table of content for all. So, this is really a way to configure the same behavior for all your documents instead of repeating yourself in all the others. And for books, like we have more formats because for books, you can do EPUB or PDF.
Wrapping up
So, that's all for the tour I wanted to give you with this presentation. It was aimed to give you a test of what Quarto can do and a big overview of the different feature, especially if you are a modern user and you haven't tried Quarto yet, you should really like try it out, see the new things. And so, if you want to learn more into all this, we have in the website, getting started section, like if you are R user, Python user, or even like a terminal user, like you have different getting started workflow. We have the all user guide, which details examples and features on the format, things like that. And also, Quarto is a community repo, community contributed repo with a lot of examples. So, extension that I mentioned about, but also like template presentation example, like anything. So, I hope it was okay. If you have any question, you can reach us on our discussion in the Quarto dev repo. So, don't hesitate to open a question or an issue and give us feedback, like the product will evolve with all the feedback we get. Thank you.
