Quarto Live: WebAssembly powered data science learning

Transcript#

This transcript was generated automatically and may contain errors.

Hi, yes, my name is George Stagg , and I'm a software engineer at Posit, and today I'm going to talk to you about Quarto Live, one of the things I've been working on for the last few months. So yes, thank you for inviting me to talk about this in this session. So I've been working at Posit for about two and a half years now. My day-to-day is as a software engineer in the open source team, and I work on things to do with WebAssembly. That's my sort of day-to-day is working with WebAssembly. And I'll explain what WebAssembly is and the consequences of that a little bit later.

Okay, so yes, so what I've been working on is an extension for Quarto called Quarto Live. And before I show you what Quarto Live can do, I did want to give a quick introduction. Again, if you've already seen Quarto, just to talk a little bit about what Quarto is. You'll probably see more of this throughout the workshops, but the talk didn't feel complete without it. So what Quarto is, is it's an open source scientific and technical publishing system. So you author documents in Markdown, and it's a literate programming system, which means that you can very easily weave together programming and text and explanations in an analysis. You can weave various different languages. The main ones I know of that work really well in Quarto are Python, R, Julia, and something called ObservableJS, which will pop up again a little later.

So what's really great about Quarto is that because of the fact that it works with programming languages in this way, it can dynamically generate documents. And you get really smooth integration with your code into an analysis and an actual publication output. And that involves even dynamic things. So it supports things like Jupyter widgets. It supports Shiny for Python . And its real focus is on this idea of analysis, publication, and reproducibility, the idea that you can take your Quarto document and reproduce it on another machine very easily.

Quarto's output formats and extensibility

So I like Quarto because it has some really beautiful output, really high publication quality output, and in a variety of formats. I'm going to focus today on the HTML outputs, but they can do PDFs, they can do Word, EPUB, they can do all sorts, including slides as well as documents. So the slides you can see on my screen now, Quarto slides, so the source code that built this presentation is all code. And I'll show you some of the things I'm going to talk about today directly in these slides because of that. So it's running in a web browser. Because of that, your content is very easy to share. You can put it on GitHub, you can put it on a free service called QuartoPub. But most importantly, I like Quarto because it's extensible and it has a plugin system that allows you to extend Quarto for your own purposes. And that's where a lot of the work I've been doing on Quarto is located in extensions.

So I've got some links on the screen here. They take you to a list of extensions and there's plenty to choose from. Because it's a free and open source system, anyone can come along and build their own extensions. And there's sort of curated lists that you can find online of extensions that you can add to your own Quarto project. So one of the nice things about the HTML output of Quarto is that because it's web-based, you can interact very easily with other web technologies. As an example, here's a web technology that will do fluid dynamics directly inside your web browser. So you can actually get kind of an approximation to Navier-Stokes fluid equations. This kind of very interactive dynamic experience that makes something like learning so much more interesting can be enabled by web technologies.

Running Python and R in the browser with WebAssembly

And that's not the only web technology. Another web technology is something called PyDyed. It's a way of running Python code directly inside your web browser. And I can show you an example of that. Here I've opened this console window and I can type some Python code and get some results directly. This can be more complicated stuff, bigger numbers. Let's do a power, a really big power. Oh, no, I've crashed my browser. That was silly. It's actually going to try and do it. Sorry about that.

Let's try that again. I want to do a smaller power. There we go. Right. As well as running in a console like that, as long as you don't blow it up by trying to calculate infinity, you can import Python packages. You can even do plotting. So I do import.pyplot as PLT. Then PLT.plot. You can get results directly on the screen.

Okay. So these are kind of simple examples. And the point here is not really the examples themselves. It's that this is running without having to have Python installed on my machine. If you were running this on a completely vanilla machine that just had a web browser, this would work. And not only that, it doesn't require any execution service. So you don't need any setup. You don't need to have a box running in a server room. You can put this on something like GitHub and it will work. So a student who's brand new to programming could load this page and run Python code without any kind of setup. They can get up and running really quickly.

So a student who's brand new to programming could load this page and run Python code without any kind of setup. They can get up and running really quickly.

Pydive is not the only way to do this. And there's also the same type of technology available for other languages. So I'm the lead developer of a system called Web, which does the same thing, but for R code. So you can run R code inside a web browser.

And the way this works is a web technology known as WebAssembly. So WebAssembly has been around since about 2017. And it's becoming more and more integrated into projects as time goes on. Over the last couple of years, it's become very popular to add WebAssembly support to your projects if they would work well on the web.

And what it is, is to give it the technical name, it's a portable binary code format. What that means is that if you take a piece of code and you compile it for a machine, normally it would only run on that machine. So you can take something like Python and compile it for a Mac, and then it won't run on Windows, but it will run on your Mac machine. The difference about WebAssembly is that it will run on those different types of machines from a Mac, Windows, Linux, a phone, a tablet. It doesn't matter. As long as it has a web browser that supports WebAssembly, it will run your code. And that's what portable means in this context. And that makes things much more reproducible and easy to get into because you can take this piece of code that's been compiled and run it anywhere very easily. It's very high performance. It's almost as fast as running a traditional program on your machine. And it's also supported by most modern web browsers. Chrome, Firefox, Safari all support it. Edge, it works as well. And basically most modern web browsers will be able to run WebAssembly code.

Introducing Quarto Live

Now, it's interactive through JavaScript. So if you know JavaScript, you can use this technology, this WebAssembly technology. But not everybody who knows Python or who knows R knows how to use JavaScript. And that's where things like Quarto extensions come in. So my extension is Quarto Live. There's a link to a GitHub website here. So you can actually view the source code. It's all open source. It's all free. You can do what you like with it. And if you add it to your own Quarto project, it will allow you to use this JavaScript technology in your own documents and reports. Documents and slides. So you can have these interactive WebAssembly examples in your own documents.

So I'm going to give you a quick example of what this looks like as an author. Mainly because to me, I think this is just so easy. If you already know Quarto, using Quarto Live is just so easy. Here's an example of some Python code. That just draws a small graph there. And the trick to make this into an interactive WebAssembly block is to change two parts. You would change where it says format HTML. You change that to live HTML. And where it says Python, you change that to pyland. And that's it. And what happens is on the right hand side, we have the output. You get a nice interactive code block where you can modify code. And the learner or the user of your output can hit run code and actually get those, view that output. And again, this is completely dynamic. I can change things. So if I'm a learner, I may think, okay, well, now I know what the difference between sine and cosine is in this graph. What does the 2 pi here do? Well, you can change that. You can have a look. Okay. What about 100? And you think, oh, well, why is that spiky? You can play with all these parameters. You can really get a proper feeling for the Python code in a way that you might not otherwise be able to do so with a single example.

Now, people who are seeing this in the Python ecosystem might immediately think to themselves, these are Jupyter cells. And kind of. They're similar in some ways and different in others. To me, Jupyter is a tool really very good for exploration and for discovering things. While Quarto Live is more to do with publications. Once you've discovered a result or the result of some analysis, you can create these Quarto Live cells to be more of a publication of that result with just a little bit of interactivity. If zero is pure code and one is a publication, a fixed publication, I would put Jupyter at about 25, 0.25, and I'd put Quarto Live at about 0.75. They're kind of flip sides of the same idea. At least to me. That's how I picture it.

To me, Jupyter is a tool really very good for exploration and for discovering things. While Quarto Live is more to do with publications.

Interactive exercises, hints, and feedback

As well as interactive examples, like I just showed, you can also create exercises. Now, I won't go into too much detail about how this is working, but I know James is coming on later to give more information about these kind of exercise examples. So I'm sure he'll go into much more detail. But the basic idea here is that similar to before, you can create these PyDive code blocks. And if you mark them as an exercise, you can make interactive exercises for your learners. So here, for example, Quarto Live knows that if you have a string of underscores, it's a blank that needs to be filled in. You get a nice message saying, please replace the blank with code.

One of the nice things you can do here is set up a code that's not shown to the learner, but is run in the background. So here, for example, if I have an example that says, OK, plot this data inside the data from data one, you don't want to have to confuse things by asking the student, OK, run this piece of code to read this data from disk. And it might be easier if it was just there for them to use. And that's the idea of these set of blocks. You can create blocks that are run automatically. And in this case, it's loading some data off disk for the student to be able to interact with.

Now, we also might want to provide a little bit of help for a learner. If they're brand new and they really don't know where to start, you might want to provide a hint. And again, you can write just standard Quarto syntax. The only real difference from a normal piece of Quarto markdown is that you're linking it to a Quarto Live exercise. And what Quarto Live will do is it will take that hint and it will hide it until the student asks for it in the UI. So here you can see there's a show hint button. And if the student clicks that, they see that hint. And the same for a solution. You can write a standard Quarto markdown, you know, demonstration of some code. And when you mark it as a solution to be linked to an exercise, it will only be shown when the student asks for it.

What's really nice about this is that because this is normal Quarto markdown, you can use all of the tools that Quarto provides to make great content. Tools like callout boxes that can be collapsed. Tools like annotated pieces of code where you can explain what each line does. You can add all of that content into your interactive exercises with very little effort. They just almost transparently work with Quarto Live in this way.

Here's the result. You have an exercise block. This doesn't have to be in a presentation. This could just be in a standard website. I can show a solution. And if I just copy that solution, paste it in, you'll see that. In this case, the data that I stored shows a picture of a dinosaur. A lot of students these days, they're not using laptops like what I'm on now. They're using other devices, mobile phones. I used to be an educator myself. I worked at Newcastle University for a number of years before I worked at Posit. And even then, this was a few years ago now, even then, a lot of students were doing their work on things like mobile phones. And I'm told it's actually becoming more common even since then, I've been told by other educators. And for that reason, Quarto and Quarto Live are both designed to integrate really well with this kind of form factor. So you can see Quarto itself has sort of shrunk down to a, has reacted to the small screen size and shrunk down the column size. And Quarto Live has sort of hid parts of the elements and sort of tried to make this as nice as possible in these constrained environments. And luckily, because we're using WebAssembly, even these low power devices can run the same piece of code that's been compiled into that format. So you can have all of these examples working even on these small and power constrained devices.

Custom feedback and Observable integration

Okay. I want to show one last example of creating custom feedback algorithms. This is really nice because you might have a code exercise and you want to give feedback to your learners. You know, there might be a common mistake that is made and you want to provide specific feedback. So if you, you can create these blocks that are marked as check blocks. And these are checking algorithms. What will happen is it will take the student's answer and it will run a piece of Python code to decide whether the answer was correct or incorrect. And you can provide feedback in the form of a list. So here, this is sort of a simple feedback that just checks that you've given the right answer. So if I say sum n, that's the incorrect answer because it's asking for the average. And it will say, sorry, that's incorrect. And then if I divide that by the length, it shows, great, that's the right answer. And again, if there was a specific situation that you were expecting students to make a specific mistake, you could catch that in your marking algorithm because it's just Python, right? You know how to write Python. You don't have to, you know, you don't have to mess with anything proprietary. You just write a piece of Python code to catch that specific situation and provide specific feedback.

One of the things that Quarto can do is integrate with Observable. What Observable is, or Observable.js in this case, it's a reactive variant of the JavaScript programming language. So if you know JavaScript, you can hook these sort of web-based inputs into outputs. Now, this is an example with R rather than Python, but the same principles apply, where you can create these interactive, responsive environments without having to use something like Streamlit, without having to use something like Shiny or Python. Observable itself is its own reactive environment, so that you can hook Observable up to these WebAssembly engines, PyGuide and WebR, using Quarto Live. These can build very, very sort of engaging experiences, and it's this system that powers the marking algorithms that are shown on the last few slides.

Okay, so that's a really, really quick overview of Quarto Live and the idea, the idea of bringing these WebAssembly-powered code blocks and exercises into your Quarto HTML documents slides and reports. Like I say, it's open source, and there's links to documentation as well, so if you want to learn more, you can do that by reading the documentation. That will show you how to install it, how to use it, how the syntax works and all of that. And with that, I'll leave you there, and I hope you're excited for the workshops later in today's sessions. Thank you.

Quarto Live: WebAssembly powered data science learning | pyOpenSci

Transcript#

Quarto's output formats and extensibility

Running Python and R in the browser with WebAssembly

Introducing Quarto Live

Interactive exercises, hints, and feedback

Custom feedback and Observable integration

Featured software#

Quarto