Workflow Demo Live Q&A - March 27th
Q&A session from the March 27th Workflow Demo: Elevate your Quarto reports with styling and scheduled emails. Demo recording: https://youtu.be/Hl9TvhlzfV4?feature=shared Resources shared in Q&A session: Community discussion for ongoing Quarto questions: https://forum.posit.co/tag/quarto Quarto document language: https://quarto.org/docs/authoring/language.html babelquarto (for multilingual project, book, or website): https://docs.ropensci.org/babelquarto/ Quarto Manuscripts: https://quarto.org/docs/manuscripts/ Managing Execution in Quarto: https://quarto.org/docs/projects/code-execution.html Quarto Extensions: https://quarto.org/docs/extensions/ Project Profiles in Quarto: https://quarto.org/docs/projects/profiles.html Custom branding deeper dive: https://www.youtube.com/watch?v=V82BBU9ldcM Quarto Parameters: https://quarto.org/docs/computations/parameters.html Lua Development: https://quarto.org/docs/extensions/lua.html Quarto CLI Discussions on Github: https://github.com/quarto-dev/quarto-cli/discussions Data Science Hangout every Thursday at 12 ET: https://posit.co/data-science-hangout/ Demo: 11 am ET Q&A: ~11:30 am ET
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hi, everybody. Thank you so much for joining us today for the demo. And thank you all so much for joining us for the live Q&A. Let me pull Christophe on over here. Thank you for the amazing demo. As a reminder, I told everybody in the beginning of the demo, there's a few ways you can ask questions, but just want to remind you here, feel free to just ask questions right in the YouTube chat. But you can also ask questions using the Slido link that I share on the screen. And we can repost that in the chat here for you as well.
But I did want to just before we dive into the Q&A, I did just want to share a few things with everybody. Well, first off, just wanted to make sure everybody knows that registration is now open for Posit's conference in August in Seattle. And so I just want to make sure that everybody knows that that's open. And I'll share the link to the conference site here in the chat. And then one more thing I just wanted to share, sometimes people ask me about ways to keep up to date on events that we host here at Posit. Also about blog posts and everything that we put out there. And I'm not sure this subscription management form is always so easy to find. So I just wanted to share that in the chat as well. So if you go there, you can select which items you want to keep up to date on like events, for example. But thanks again for all the great questions that have come in so far.
Before we dive into those, I think it'd be helpful maybe to just introduce ourselves real quickly here. If I haven't met anybody yet, very nice to meet you. I'm Rachel Dempsey. I lead customer marketing here at Posit. I help host these workflow demos every Wednesday. So the last or sorry, the last Wednesday of every month, but also our data science hangout, which is every Thursday from 12 to 1 Eastern Time. And I'm always joined here by Ryan. So I'll turn it over to you to introduce yourself as well.
Thanks, Rachel. So hi, everybody. I've probably seen my face in a few of these demos. I've hosted a few in the past, but I usually just kind of hang out in the Q&A, just trying to help out with some of the questions. But I work here at Posit as a data science advisor. So just making sure folks that are using our tools, both open source and professional tools are getting the most value out of them. So I do a lot of workshops, webinars and demos like we do today. So it's really nice to see everyone here again. And again, thanks for all the questions. And we'll get to those here in a second. But let me actually just pass the mic over to Christophe again to reintroduce himself.
So hi, I'm Christophe Dervieux. So I'm working as software engineer at Posit. I'm part of the Quarto team, but I'm also working on the R-Magnon ecosystem. So all the R packages, but like most of my work also today are on Quarto on that. And so thanks for joining the demo. I hope it was clear.
Custom styling with SCSS
Great. Well, let's dive in. So I see there are a bunch of questions over on Slido already. So I'm just going to pull up a few of them. And let's start with this custom styling question. So there was a question for custom styling. Where did you get the starting dot SCSS file that you were customizing?
So for the demo, like I understand, like I did like copy paste, but it was like for an example I created just for the demo. But as a start, so you could find this file in the source repo that is open. So this will be the same file as I used for the end file, but you can navigate into the tree and the tag in the repo if you want like to show the first one. I copied to recreate the demo, but otherwise in our doc, the Quarto doc, in the Quarto seeming theme pages, you will have example of a very like small SCSS file you need to run. So what you need to know for this file is the specific like rules that you need to put so that the layers are correctly handled by Quarto. So you'll find an example in the doc and also in some of our templates when you create a website, things like that, you could have a look at it.
Awesome. And I shared the GitHub repo in the YouTube details for the recording, but also just put it in the chat right now as well. Ryan, if you want to ask this next question, I know you have a helpful resource to point us to as well for this one.
Yeah, absolutely. So this is another good question that came in. I believe this one came in from the YouTube chat. But what if I want to create a report that matches my company's specific branding? So things like font, colors, logos, banners, what would that process look like? I'll actually I'll hand it off to Christophe here in a second, but I do want to repost that we'll put it here in the chat that a few months ago now, we've been doing these for a year now, I think now, Rachel, which is pretty crazy. But a while back, we actually had a session that kind of introduced you to custom branding for Quarto and a lot of stuff that Christophe showed today as well. But check out that recording. Again, all these demos are recorded and hosted on YouTube. So you can always go back and check them out. So that'll be a good starting resource. But Christophe, anything else you want to add there?
Yeah, Quarto is evolving quite fast. So I guess like months ago, it was like an old version of Quarto from today. So we improved. The challenge, the demo was meant to be for companies branding, company styling, where you need to follow use palettes. But I understand like fonts, logo can also be added. So depending on the format you want to use, it will be easy or less easy. And so this is ongoing work. So we'll need to improve. It's part of the roadmap to improve like the styling, the brand styling, so that you have one way to configure that will spread across different format. But as of today, the way of doing that would be to use the SCSS approach, the Quarto seeming system that I showed in the demo for HTML. If you want to do that for other formats, you will need to use a specific configuration for those formats, for LaTeX, for TIPS, or for Reveal.js. So HTML is based on Bootstrap, for example, Reveal.js is not. So some of the variable will be like common, but some of them won't be. And so it could be some adjustment. So it's part of the work to make that easier. And then I think it's part of the question, if you want to share that across your company with several users, you need to look at Quarto extension. This would be like the keyword to look in the doc, how to create an extension. And so you would create, so I didn't show that in the demo because it was like one step further, but you would create a custom format in which you would put your branding assets and style and things like that, and make an extension so that other users can Quarto add your extension, which would be your scene. And so you would add everything into there instead of just like sending assets to anyone who wants to use.
I put this into the chat as well. But if anybody has very specific requirements for this, and this is something you need for your company, I know the Quarto team was interested in seeing a few like company brand docs or company brand guidelines. And if you would be open to sharing that with me, I put my email in the chat as well.
This is definitely something we will work on, we want to work on. And so we have our own example of branding, but having more example or having feedback even in our Quarto discussion place on our repo, like having feedback on what you would like to do, want to do is always useful to drive our development.
Logos, banners, and format-specific styling
Perfect. Let me pull a few questions over from YouTube. And I see Ben just asked, what are the dot scss variables for adding logos, adding banners? I've tried to find them, but there are so many.
Yeah, so logos is something that is quite specific to formats. And so it may not be a CSS variables available. So for example, the way to add logo in HTML document would be a bit different from what you could do in other formats in reveal.js. So some of the format will have a YAML variable for logo, which is integrated, like if you do a Beamer presentation, but like in company context, usually you don't use Beamer that much. So it's part of what we want to make easier with the branding mechanism. So I can't point you right now to one specific variable. And it's the same for banner. Banner will work. You can customize banner for website, for example, but a single HTML document won't have a banner by default. So if your template in your organization needs to have like a banner in each HTML document, then you will need to tweak like more easily than just a CSS styling.
Combining multiple Quarto analysis files
Yeah, I was actually just reading this question. It's a really good one. So how would you recommend creating a single report that needs to communicate results from different Quarto analysis files, so potentially using the embedding feature? And I don't know, Christophe, maybe this is a good use case for the relatively new manuscripts feature that came out. But maybe that's something we can chat about.
Yeah, so the embedding feature is part of the manuscript feature, but it can be used outside of the manuscript format. So it's meant for that, like to have notebooks and to share results. So it's one way to do that, but it will work only for sales and sales output, specifically pointing to the notebook. But if you want to reuse some content, like some content you write, you can also include some part of the document in several QMD files. So this would be a way to have like some includes that would be shared in different analysis. And this could be also used to build like a main report from several reports, but it's a bit more organization. So the most built-in feature would be the embedding feature. That's the right place to look for that. It's a recent feature, a new feature, so that could be improvement. So don't hesitate to give feedback. And otherwise, it would be like workflow tool, I would say, that can be integrated with Quarto, where you would like probably render your like QMD analysis that would produce some plots, data that you would save. And then after that, you would render the main report that would use those computation results.
Multilingual reports
I think if you are using projects, you should, it should work. Or maybe, yeah. But you need to use like book project. Like if you use a book project, you should be able to set the word as an output format. And it should like have a single file in the end. If it's not working, it's a feature we should like develop. It's working for PDF for sure. So you can like for a book, you can have like a book website or a single PDF, which is in several QMD source. So I would say it should work for Word. But it's specific to books. You can't have a website output as a single Word. It's not the same project. You need to have like two project or profile feature to have like two different rendering.
Thank you. Another question that was over on Slido is, is there a recommended or maybe this was from YouTube earlier, is there a recommended for multilingual business reports with Quarto, maybe with auto translation for HTML output? Is babelquarto the way?
So for those who don't know, babelquarto, which is mentioned is R package. So it's tied to Quarto in a way that the person who did that wanted to create several books in different language with an automated way of doing that. And so the developer used R to like organize this, organize the Quarto rendering. So this is babelquarto. So it's not the Quarto way in a sense that it's not made by us directly, but it's a way to do that that is like provided by the community. But Quarto has interesting translation feature, we have like all language field and language fields. So we have auto translation for anything that Quarto will write for you. So we have templates, HTML templates, for example. And in those templates, we can use some English word in the default. But if you change the lang attributes, this will be translated using default translation that you can customize. But it won't do auto translation of your own content that you write. So currently, there is no way automatic way to do that. You would need to write yourself several, several production. And we don't have yet a way to organize several books, for example, in different language into the same output. But this needs to be done with an outside workflow like it was done in babelquarto for now.
Code-specific questions and community resources
Yeah, so I just noticed a few of the questions coming through here, which are really great questions. But if you have a specific, and I call them like code related questions, so maybe you're trying to run something in Quarto, and you're seeing some error message, it likely requires a little bit more context on our end to kind of see what that code is, what's the actual error message. So what's really helpful for us and really the entire data science community is if you post those questions to our community page. So we just link to it right there in the chat. And we actually have a specific tag for Quarto. And again, that just opens it up. So here we have Posit can try to answer it. But really, anyone in the data science community can use those answers. And the more context you can provide in terms of what you're trying to do, error messages, the code that you're actually executing really helps us to help troubleshoot your bugs.
Custom styling for PDF vs HTML
The so the custom styling I showed in the demo is based on SAS, which is a tool for CSS. And so it will work for HTML only. So it will work for the default HTML format, but it will also work for reveal.js, where we will like know how to handle the Quarto theme. But for PDF, it won't work exactly the same. So any styling that you can do using YAML variable, they could share the same configuration. For example, you will have font configuration variable, this would work for PDF and HTML. But anything related to Quarto theming, as we call it with a specific SCSS file will only work for HTML formats. But the brand thing I mentioned before was is to find a way to design something that would make possible to have like some styling configuration work for different tooling. But like, you can understand that each tool is styled differently. And so the challenge is to find like, a common way for like have one input to translate to different inputs. So we are experimenting and working, working on this, this type of thing.
Caching and freeze in Quarto
So there, as it's referencing R Markdown, I will like there is several caching understanding in Quarto. As you are referencing R Markdown, I will say that we have the cache option for the computation. So you have a caching for the computation with the cache option. And this can be set for Jupyter engine and Nitar engine. But in each of the computation world, they will use, it will not, it's not a new caching mechanism. So if you use this caching for a document using Nitar, it will be the same caching that in R Markdown. No difference. So it will be as granular as in R Markdown, because it will be Nitar doing the caching and it works exactly the same. If you are working in a Jupyter document, engine document, it will use a caching that Jupyter provides. The same. So it could be different from like the Nitar one. So I don't know exactly the granularity if it is the same. But the caching is, will run the computation and use a caching mechanism. But in Quarto, you have also something that we call freeze. So the freezing mechanism, which is a way to tell your Quarto project to not run computation in certain condition. So it's a bit different than caching because at this point, no computation will be run. It means like if you freeze a website, you don't even need after that R or Python to render your website, your output. So I will put a link so that it will be shared. So I won't explain this in detail here. But this can be useful in some situation. And so this one is not granular at all. It's really global for the project, the Quarto project.
So it's a bit different than caching because at this point, no computation will be run. It means like if you freeze a website, you don't even need after that R or Python to render your website, your output.
I just forgot about the reticulate part of the question. This is so this quite specific to our users. But this is the same caching as in Nitar. So I believe that reticulate has caching support for Nitar. And so if it's not working like perfectly, this is even like not a Quarto thing or Nitar thing. It's a reticulate thing because Nitar will like ask reticulate, how would you cache your Python environment? So it should work. It should work, but it's not Quarto specific at all.
Embedding Quarto reports in SharePoint
Thank you. So this one from YouTube was, can you embed Quarto reports into SharePoint sites using iframes? I had trouble embedding Shiny apps in SharePoint using iframes.
Okay, I can relate. Because like in my past in industry, I used like SharePoint. So I don't know exactly where we are at, at SharePoint, like evolution on how it works. But like, it's really a SharePoint issue. What Quarto will produce in document or website will be static documents, which will be HTML document with resource. So you need to put that in your SharePoint environment. And it should not have any limitation to load those files. But Quarto is not aware about SharePoint and what is produced by Quarto can be put in any server, web server that can like serve those kinds of files. So the difficulty can come like the trouble, like Shiny is a bit different because Shiny needs R to run. But when you produce a Quarto document, it's like just HTML file, CSS file, PDF file. So you need to have all the resource or produce like embed everything into the output. So I would say it should work. But it's mainly depending on the setup of the SharePoint server in your company.
It's a bit. It's a bit different. Like the work that was done with Confluence is that you can publish directly a QMD file to Confluence using a specific, it's using a specific Confluence format and an API on server. So we, concretely, we translate the QMD results output is XML file that is sent to Confluence. Technically, it's that. So it's not exactly the same. You really publish to Confluence and you are not just posting HTML content or HTML output to Confluence. But this is not the same for SharePoint. And I don't think SharePoint has a specific format. But I'm not exactly aware of that. So it's a bit different. But you can publish to Confluence.
Quarto books and multiple output files
I was just thinking again about the manuscripts feature, which honestly, I haven't played around with too much. But I know it's a way to at least take multiple Corda documents and combine them into a single doc. But like Christophe mentioned, you can do the same thing with Corda books and websites.
The manuscript format is quite tied to design at first for academic publication. And so I think it works well for HTML and PDF output, which more feature on the HTML side of things. I don't know for Word output how manuscript feature really behaves because like embedding would work a bit different, for example, but probably you can leverage that. But it's really meant for having several QMD documents, but only needing some results part. Like you have analysis and you want to make like a paper or report, like, but refer to analysis. This is a clever way to like access the results and have a link direct to the notebook that you are doing. So it's another use case.
Scheduled emails and Posit Connect
Currently, like the Quarto creates the email preview as an HTML file as an output. But the fact to be able like to send this as an email automated, like with a report is a Posit Connect feature. And so currently, the implementation is tied to Posit Connect. And so that potentially that could be evolution where we could produce an output to be sent to another, I don't know the term, but like SMTP server or something like another tool. But like currently, like we produce the feature and then the Connect team has done the work to like understand the output and like use what we produce to connect to the email feature. But there is no way in Quarto to like have the Quarto send email for you. So but I don't know if the question was like that part or, or the output format. But currently, it's a Posit Connect only feature, like in the way it's used. But what we produce is produced by Quarto CLI itself.
Snakemake and workflow integration
Yeah, absolutely. So can Quarto be integrated with Snakemake for an end to end workflow management and report generation pipeline. And so for those that maybe aren't familiar with Snakemake, it's a way to essentially create these automated workflows where something depends on something else. And so if you rebuild something, it kind of automatically rebuilds the dependencies for that project. So there's a lot of tools, either CLI tools like Snakemake, there's also some built in R. So if you've heard of something like the targets package, it kind of does a very similar thing. But I'll let Christophe kind of speak more to that.
I don't really know Snakemake. I know the term of the category of tool it's using. So it won't be a precise answer related to Snakemake. But what I can say is that Quarto is a CLI tool. So it's really like a system program like that you install and exify with some resources. And so it can be integrated in any tool that can run some commands. So Quarto itself is not integrated with such tool, but you can integrate Quarto in your workflow probably to help with this kind of process. There could be some interesting feature in Quarto, which are the pre-render and post-render script that you can have in a project that could be useful. I know, like you mentioned, our example of targets, they did integrate Quarto in there. So they have specific way in their workflow to detect Quarto document, know that it needs to be rendered with Quarto. And so they are calling Quarto render. And so they are using like some of the feature. And so if some people in the community try to integrate Quarto with Snakemake and you are missing some information on some feature, just open an issue, a feature request, and we can add anything that helps making those integration. We won't make those integration ourselves, but there is, for example, a Quarto inspect command, which is not for user, but for people like needed to have information for such projects. And it's not perfect yet. And so probably we are not outputting everything, but if something is missing, like you can ask. So I would say it can be, but like, you need to work on it to make it happen.
Parameterized reports
So there was a question, I noticed you set up params outside of the YAML. Is that the preferred way to do that in Quarto?
I used, like, it was quite specific to the demo. So I use the term params, but like, it just was like a value in document where I said something, but it was to show that you can easily make that a parameter of the report using the parameter feature that Quarto has, which is like, if you are using a Nitar engine, it's tied to how it works in Armageddon. If you are using the Jupyter engine, it will be tied to how Peppermill is working, which is a tool to parameterize IPoWNNB reports. So the way to parameterize a report is to use parameters. So I set up param outside of the YAML here, but my document was not parameterized. I would have a need to, like, modify the source to re-render a different version of the report in the demo that you saw. If you are using parameter, and so you put what's needed in the YAML header, or all the specific chunk for Jupyter, then you'll be able to use the same source file and call Quarto render with some parameters, and it will be a different output with the same source file. So this is a bit different. So if you need that, then you need to use parameter. But if your parameters are coming from, like, databases, the data, things like that, you may not need parameters.
I did not get the point how to make sure a parameterized report would send to its corresponding recipient. So is there, would there be a way to do that with, like, just sending to specific people based on the parameters?
The sending part is really a Posit Connect feature. So, for example, I'm not sure as of today, parameterized Quarto documents are supported in Posit Connect. This would need to be checked. I'm not sure, like, it has been integrated yet. So you need to find, like, other ways to make a report scheduled using information. So it's why you didn't see that in the demo, and it may not have been clear. But regarding the recipient, it's really, like, a Posit Connect feature. So I don't really know what you can customize in there, if you can customize recipient or not. But again, like, email is a Quarto 1.4 feature, which was just released. So it's the first version of it. So having use case feedback and also, like, for our pro customers using Connect, we will be able, like, to add more feature. And so it needs to be, like, joint work between, like, the Connect team and the Quarto team to make that happen. But that is interesting. So interesting question. Thank you.
Lua filters and CI/CD
Yep. So I just want to come through. I'd like to know, is there Lua filters for applying customization for this kind of report? I know we have a lot of great docs on the Quarto site about using Lua, so.
Yeah. So for context, for anyone who don't know, like, Lua is a specific language used. That's why I made Ryan ask that question. I wasn't exactly sure what Lua was. And if I was going to say that, right?
Yeah. So Lua is a specific language that is used, like, usually in some specific fields. And so in Quarto, this is related with Quarto because this is the best way to extend the behavior of Quarto. And we mainly use this because this is the extension language for Pandoc, which drives, like, Quarto rendering. So the question is about example, about, like, using Lua and so extension for customization. So I believe this kind of report is, like, starting report. So you can do almost anything in Lua. We have documented that quite extensively in Quarto, also because, like, we are, we have created, like, a Quarto Lua API. So some function in Lua that you can only use with Quarto. So you should, like, the best way is to go into this part and, like, read the doc, learn about Lua, how it works, and then understand that it's quite advanced because you need to understand, like, what will happen. And in Quarto 1.4, we have improved that also because you are now able to, to insert your Lua filter in specific place in the, in the rendering of Quarto. So access, but, like, almost all the feature you can see of Quarto is done internally with Lua. So you can do a lot of things. So I can't point you to an example for this kind of customization. But you can maybe find some in the discussion boards on, on Quarto dev, Quarto CLI repo, because some people share sometimes this kind of examples or report and there is gallery of example on the website.
I'm not sure. I'm not sure to understand the question, embed CI-CD pipeline. Like, you can use Quarto in CI-CD pipeline. But as your other question is embed in Quarto, I would say no, but I'm not sure to understand. So the best thing for you, Jeff, would be like to ask this in the discussion board of Quarto CLI repo so that we can, like, discuss there. And so the discussion board is made for discussion and question, not just issues. Like, we have the issue board for that. So feel free to ask questions there also.
There are so many awesome links shared here. So I will go ahead and go through and consolidate from the chat and then just put those over to the recording as well so that everybody has it available because the demo recording is made available immediately after it ends as well. So you can go back to it and also use the GitHub source code that Christophe has shared with us, too. But thank you so much, everybody, for joining us today. Thank you so much, Christophe, for the awesome demo as well.
Thank you for having me. Thank you, everyone.
If there are any other questions or something we didn't get to, or if you're curious about trying out tools like Posit Connect, you can also use this link on the screen here to book a call with our team. So I just wanted to make sure I shared that there as well. Also to once again remind everybody that registration is open for the Posit conference in August, and we'd love to see you there. While I have everybody, why don't I throw everything at you? I also host our weekly data science hangout, which we have every Thursday from 12 to 1 Eastern time. And so each week, a different leader from the community joins to just answer questions from you all. So it's a very casual, informal time to just hang out with others from the community. And tomorrow, Zach Fragoso from Domino's Pizza will be joining us. And so I can also put the link to the data science hangout in the chat here, if anybody is interested in joining us and spending more time with us all. But thank you all so much. Have a great rest of the day, everybody.
