surveydown (John Paul Helveston, GWU) | posit::conf(2025)
surveydown: A Markdown-Based Platform for Interactive and Reproducible Surveys Using Quarto and Shiny Speaker(s): John Paul Helveston Abstract: This talk introduces the surveydown R package and survey platform, which leverages the Quarto publication system and R shiny web framework to create reproducible and interactive surveys. While most survey platforms rely on graphical interfaces or spreadsheets to define survey content, surveydown uses plain text (markdown and R code chunks), enabling version control and collaboration via tools like GitHub. It supports complex features like conditional skip logic, dynamic questions, and complex randomization as well as a diverse set of question types and formatting options. The open-source package gives researchers full control over survey implementation and data storage, with reproducible workflows that integrate with R data analysis. GitHub Repo - https://github.com/surveydown-dev/surveydown Materials - https://github.com/jhelvy/2025-posit-conf-surveydown posit::conf(2025) Subscribe to posit::conf updates: https://posit.co/about/subscription-management/
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
All right. Hi, everyone. So I am JP Helveston. I am a systems engineering professor at George Washington University. And yeah, I'm excited to share with you our platform that we've developed called SurveyDown. In my research, I try to figure out what people want. I study people.
Now, I don't have a crystal ball. And I am a professor. But I'm not this kind of professor. So I can't read your minds. And I have to rely on other tools to learn from you. So I rely on surveys to understand people and their preferences. I use web-based surveys. Most web surveys look something like this. You know, there's different types of questions. People can type through them.
The questions are all on different pages. This is near the end of a survey here. And chances are, if you've ever made a survey, it probably looks something like this as well, some sort of web survey with forms on different pages. And if you've ever made a web survey, you've probably used a tool that looks something like this. This is, of course, Google Forms. But whether it's Google Forms or Qualtrics or any other kind of survey software, it's usually a what you see is what you get interface. And that interface is really nice for really basic surveys because it's intuitive, easy to use. But it does have a lot of limitations, especially as a research tool.
Limitations of existing survey tools
So first of all, they're fundamentally not reproducible. If I want to recreate someone else's survey, I have to go hand make it and click and drag things and add content manually. They're also kind of difficult to version control. So if you've ever made versions of a survey, your Google Drive starts to look a little bit... Look like this, right? Lots and lots of forms and versions of those forms, and it's just not working great. They also have limited features. I can only ask questions that are provided by the software. If there's other custom questions I want to ask, it might be hard to implement. And it's not open source. Most of them are not open source. So I have limited ability to further customize the tool or understand what's going on under the hood.
But this is a session on Quarto, right? And Quarto, this is like the feature set, right? It's like all of those things are what you get with Quarto. I'm a huge fan of it. It is reproducible. You get version control with GitHub. All of that is for free because you're just editing a plain text document. It is very feature packed, right? You can run code. So anything that you can stick in code, you can run inside those documents. And it's open source. So as a big fan of Quarto and a survey researcher, I couldn't help but think this question, wouldn't it be nice if there was a Quarto for surveys, right? You just edit a Quarto doc, render, and get survey. And I asked a lot of people, could you make this for me? And nobody was willing to do it. So we decided, fine, we'll just make it ourselves. And we made a package called SurveyDown. So it's obviously a pun on Markdown here. But the idea is Markdown-based surveys. That was the sort of vision.
So as a big fan of Quarto and a survey researcher, I couldn't help but think this question, wouldn't it be nice if there was a Quarto for surveys, right? You just edit a Quarto doc, render, and get survey. And I asked a lot of people, could you make this for me? And nobody was willing to do it. So we decided, fine, we'll just make it ourselves.
How SurveyDown works
So in my talk today, I'm going to try and walk you through what it is, how does this package work, what are some of the cool things you can do with it, and then where we're going next. We have some further developments this past summer even that we've been working on.
So what is it? It is an R package. And in a nutshell, it tries to take Quarto files and render them into surveys. So that's the gist of what SurveyDown is all about. So I think the easiest way to explain that is to just show you a really basic example. So this little survey you see on the screen here that just has a header, one question, and an X button. If you wanted to make that, this is the file you would need.
All right? So this is just a normal Quarto document. There's nothing really special about it. It has a YAML at the top. We're rendering to HTML. We have some settings so that you don't see messages and things like that. You have the SurveyDown R package that you need to load. And then the rest of this is the content that you're going to see on that survey. All right? So pages are defined using the Quarto fence thing, right? Those little three colons. If you've ever used those to control where things are displayed in a Quarto doc, we kind of took advantage of that. And each Quarto fence has this SD page class, SD there being for SurveyDown, with a unique ID. All right? So everything inside those fences, that's what's going to be on that page. And of course, you can have multiple pages on your survey.
So for this little survey, we're going to put that welcome message with a header level one, that's the little hashtag, and then we have a code chunk where we're defining survey questions and a next button. All right? So that stuff that's highlighted right there, that's what you're seeing in this rendered page on the right. So that is the gist of how we define surveys using SurveyDown.
I want to highlight one more little small thing here, which is that every call to SD question, that's our main survey question function, has an ID and an option list. And so that ID that says has fave hero, that's going to be the unique ID for this question. It's also going to be the name of the variable that you get in the resulting data. And then the option list, you'll see that there's a name and a value. And so the name, the capital Y for yes, that's going to be what's shown in the survey. The lowercase y for yes is what you get in your data. So I'm just highlighting that because you get to control everything about your survey, including what you see on the survey as well as what the data looks like that you get back, which is something that we were very intentional about.
Okay. So if you've used Quarto, you know that it renders to static pages. So you might be wondering how in the world do we go from static page to web survey? Well, we just needed some sort of web framework in R. It turns out there's been one that we've had for a very, very long time called Shiny. And so we thought, why not just use that? And that's what we did. So this picture I showed you earlier saying that, you know, we're just taking Quarto docs and turning them into surveys, that's not really the complete picture. The more correct picture is this. We're actually building Shiny apps. So every SurveyDown survey is just a Shiny app. It's a standalone Shiny app. And we're just using Quarto to control the UI, which is what you see in the survey. And of course, with every Shiny app, there's also a server in the background that's running all the logic.
So a complete survey is actually two files. You need your Quarto file. That's where you're going to define all the contents that's inside your survey, like the one I just showed you. And you need some sort of small app.r script that's defining a Shiny app. Now, if you've never used Shiny, don't panic. We've tried to make it as easy as possible for you. Really the main thing you need to know about Shiny is that it needs two pieces, a UI and a server. So we wrote two functions, stui and stserver. And that's it. It's pretty much a boilerplate file. You don't really have to worry about this file very much. The stui function is going to render that Quarto file for you. You don't even have to render it yourself. And stserver is going to handle all the logic about page turning and data management and stuff like that.
Data storage with Postgres
Okay. So now we have a web survey. But you might be thinking, where does the data go? People are filling out our web form, so it has to go somewhere. And we thought about this, too. We needed some sort of data back end that's going to store your responses. But we wanted to use, again, exclusively open source technologies. So we landed on Postgres as our main decision here to just go with that for now. And it's working really well. Now, just like Shiny, don't panic. If you've never used Postgres or you've never set up your own database, we've tried to make this as easy as possible as well. For one, a great starting point is to use Superbase. So Superbase.com is a wonderful website that gives you free access to a database.
And you can set it up when you log in and make a table. It looks like this. It looks just like something like a Google sheet or anything like that. You'll see your responses coming in, one row per respondent. So all you really need to do is get a Postgres database somewhere, whether it's Superbase or some other server that you set up. That's fine. Get your credentials. And that's really all you need is you need to hand your credentials to SurveyDown, and you're good to go.
So this is the last little step to make that little handshake work. You have to make a connection to that database. So we have a little function here called sddb connect. Once you have that connection, you pass it to the server, and the server will take care of all the data management for you. You'll see there's one more little function at the top called sddb config. That's commented out because you just need to run that one time. That's just to set up your configuration. So you get your credentials like your host and your password, stuff like that, store it as a little environment variable, and then basically forget about it.
Okay, so that is one of the nice things about using this sort of separate database to store your data is now you can have a workflow where you have one project where you have everything about your survey in one place, right? So you can see in my tabs here, I have the survey Quarto file, I have the app.r file, and I also have this analysis file where I'm, again, connecting to that database and pulling the data in right away. So in one place, I can design my survey and also analyze the data as it's coming in.
So what is SurveyDown? Well, it is our package, but it's a little bit more than that. It's really like an ecosystem. We've actually are pulling on a lot of other open source technologies to make this all work. We're using Quarto for designing the survey, Shiny as our web framework, and Postgres to store the survey data, and the R package is really just making all of that work together and providing the control logic.
Features and question types
Okay, so now that you know what it is, a little bit about how it works, I want to show you some of the fun features, things it can do. So it is very feature-packed. I don't expect you to read those features. Just go to our docs, surveydown.org, and you can see there's lots and lots of features that a lot of people have already added to it. I want to highlight just a few for today. First, different types of questions that are supported. We support 13 different standard question types that you would expect on most survey platforms. All of them are defined using one function, the SD question function.
So the code you see on the left side here, this SD question function, it will turn into what you see on the right. So text questions look like this. MC questions for multiple choice, they look like radio buttons. You have things like MC multiple, if you want checked boxes. MC buttons, these are really fun, especially for phones. These show up really good on touch phones, where you can put images and things on the button itself, and we even have sliders. Now, if you've noticed, if you looked at some of these and you said, those kind of look like Shiny widgets, it's because they are. We just totally hacked Shiny, and we're just using Shiny inputs as question types. And surprisingly, it works really well. When you build a survey like this, there's really no way, you can't really tell. It just looks like a normal survey question.
Conditional logic and skip patterns
So it's all Shiny under the hood. A couple other things I want to highlight are logic control. A very common thing you might want to do in a survey is be able to conditionally control something that's happening on the survey based on the response of a respondent. So something like this, where you have a question that says, do you have a favorite superhero? And only if you say yes, do you see that second question pop up. So that kind of conditional display logic, we wanted to be able to support that as like a native thing. That was an early, early goal in the package. So to do this, first of all, you have to define both questions. So we have SD question called two different times, a multiple choice question, and then a text question. And then that second question over in the app, we made a function called SD show if. And the logic here is that you have some kind of logical condition on the left side, a little tilde, and the target question on the right. So the idea being, if that logic is true, meaning they chose yes for the first question, then that second question called favhero, that's going to display. So we took this syntax, basically building off of the idea of case when, if you've ever used that function with dplyr. It's the kind of same idea. Condition, tilde, target. And of course, you can have as many conditions as you want. So you can just put a comma and have lots and lots of conditional display in your survey.
Using a very similar pattern, we can also skip people to different locations in the survey. So let's say if they chose no for that question instead of yes, let's say we want to screen them out. We want to send them to the end of the survey or a different page. You can do the same idea here. I've got my Quarto file where I have different pages. You can see the little fences. I have that first question, that yes or no question. And then I have a screen out page where there's really nothing on that page but a message that just says thanks for your time. And to implement this skip out, we have a function called sdskipif. So if this is true, we're going to skip you to another page. And it's the same pattern. If they chose no for this question, we're going to send them to the screen out page. So this just gives you a little bit of a flavor of how we've implemented some of these common types of programmatic interactive features.
Building on Shiny's ecosystem
You wanted everything to be able to be controlled using code or just plain text. So that gives you a preview. Now, one of the nice things about building on top of Shiny is that this is kind of true. Like if it works in Shiny, we can probably find a way to make it work in SurveyDown. So we have ten years of development work that we get for free in our surveys. And an example of this is we had a student who was one of our doctoral students who wanted to build a survey where they had this interactive map where people were selecting on census tracks and they wanted to record those census track choices. Well, there's this great package called Leaflet that does exactly that. So we didn't have to recreate that. We just took Leaflet and then played a little bit with it to figure out how to make this connect to our questions. And now inside your survey in your database, you're getting the responses that they were choosing. And this really was not that difficult to put together. But again, we're just building on top of Shiny.
Another example of something interactive like this is you can do live polling. So you've seen Slido, right, where you guys answer questions and you see the results in real time. You can do that with this platform. So I can ask you, who's your favorite superhero? And then as you're typing it up, inside the server, we just made some ggplot code to make a bar plot of the results. And the respondents can all see that and it will update in real time. So you'll see the bars grow as everyone's filling it out. So Slido next year, maybe use SurveyDown and save a little bit of money. We'll see.
Okay. One of the other things that's really fun about working with just pure plain text is now we can leverage large language models, right? So I can take advantage of every LLM that's out there now to build my surveys. So here's an example. That superhero survey that I have been like showing in the background, I didn't actually make that. Claude wrote that for me. I just told it, I'm using SurveyDown. Can you generate a survey for me about superheroes? And it made this. It's pretty nice. So if you want to make these sort of custom features like maps and things, you can probably figure it out with large language models.
SD Studio companion package
All right. The last thing I want to showcase is something that we built, I say we, but really my student, Ping Fan, who's here, and another one of my undergraduate students, Bogdan, who really worked on this companion package we call SD Studio. Because we know that not everyone who works with surveys knows how to write code or might not be familiar with Shiny. And so we wanted a dashboard interactive way to make surveys. So this is kind of a brief overview of where we're at with this project. It has a build tab where you can graphically see your survey. And you can also see the Quarto file on the right. So if you're adding questions graphically, that file gets updated automatically. So you're still getting that full reproducible outcome, but it's a little bit easier to make and add questions and content. It has a preview tab. So you can preview your survey both on desktop and mobile. You can see how it's going to look. And then it has this little response dashboard so you can see your data coming in. And this is just running as a local Shiny app, right? So you just open up our run SD Studio launch, and you get this little dashboard to work with.
Community and next steps
Okay, so we think this is a pretty powerful research tool, but of course with that power comes some responsibility. Right now it's just a three-person team maintaining this whole package, and we're really going to rely on everyone else to continue to grow this thing and make it better. So if you're interested and you want to try it out or you want to contribute, you can scan that code. You can go to the main GitHub where we're storing the source code, post an issue, give it a star. One thing you can definitely do is contribute a template, right? So if you try to make something that is a little bit different or interesting that goes outside of what something I've showed today, you can send a template and we will post that on our website so that other people can do it. So for example, that interactive leaflet map, we now have a template for that. So if you want to make a map in your survey, you can download that and see how we put it together and learn from that.
All right, so that's all I have for today. The source code and the docs are at surveydown.org. Of course, I'm going to be here the rest of today, so if you want to come ask questions, please find me. I'm the bald guy with orange pants. I think I'm the only one here. So thank you again.
Q&A
All right. So I think you're going to get a lot of questions afterwards because there's too many for this part. So find him after the talk. There's also a Discord channel for RegenC7, which you can also post there, and we'll make sure that he goes and looks there.
But question number one, is this compatible with security guidelines for sensitive data in research, thinking about HIPAA, et cetera? Yes, that's one of the most common questions we get, and just to explain it, I think, I mean, we have a whole security page on our docs, so go there and you can read more detail, but I would say it's about as secure as you are willing to make it. So part of, there's different vulnerability places, right? So Superbase, for example, is HIPAA compliant. It's probably the most secure, easy way to make a database, so that part is secure. Shiny itself is as secure as the Shiny developers have made it, right? So we have done as much testing as we can, but I would say just go to the docs page on that and you can see, there's an answer for different layers of potential areas for vulnerabilities.
All right. Can you speak to the differences between SurveyDown and Shiny surveys, and when you would use one over the other? Yes. So we were inspired by Shiny surveys and other attempts to make this kind of survey platform. I think the biggest difference was we wanted to build in specifically Quarto or some sort of Markdown environment. I think that's a really big difference, that a lot of the other ones, I believe Shiny survey works where you give it a CSV file of just questions and answers. And so for a static question, something that you might see on like a Google form, that's about as far as it goes. Because we're building more openly in Shiny and Quarto, I think we can expand and ask constant questions and things like that.
All right. And then the last question, since you talked a lot about free and open source tools, well, open source tools, can you deploy the survey for free on Connect Cloud and then use Superbase for the free database? Yes. I'm an academic. Did I not mention that? Everything we do is for free. So we have very low budgets. Okay. So yes, we host on shinyapps.io, Posit, Connect Cloud. We've tried both of them. They're all documented here. Superbase is also free. So you can totally do a fully free version of whatever project you're working on. And of course, you also have the ability to pay for more service if you need it. All right. Let's thank John for another round.
