Tracy Teal | Teaching R using inclusive pedagogy: Carpentries workshops lessons learned | RStudio
Talk from rstudio::conf(2019) The Carpentries is an open, global community teaching researchers the skills to turn data into knowledge. Since 2012 we have taught 700+ R workshops & trained 1600+ volunteer instructors. Our workshops use evidence-based teaching, focus on foundational and relevant skills and create an inclusive environment. Teaching the Tidyverse allows learners to start working with data quickly, and keeps them motivated to begin and sustain their learning. Our assessment show that these approaches have been successful in attracting diverse learners, building confidence & increasing coding usage. Through our train-the-trainer model and open, collaborative lessons, this approach scales globally to reach more learners and further democratize data. About Tracy Teal: Executive Director of The Carpentries (https://carpentries.org) and co-founder of Data Carpentry (http://www.datacarpentry.org), a non-profit organization that develops and runs workshops training researchers in effective data analysis and visualization to enable data-driven discovery. Manages projects, operations and finances. Leads lesson development and volunteer coordination and is responsible for strategic and business planning
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
I'm really excited to be here today to have this opportunity to talk with you and be invited to give a talk. And so first I want to say I get to be the one here to give a talk. But what I'm going to talk about today is the contributions of many people that make all of this work possible and drive the energy and enthusiasm of this project and so much of what we talk about.
I mean, so just in this room, contributing to what I'm going to talk about today, we have Mine and Jenny and two of our staff here, Kerry Jordan and Francois Bicheno. We have Greg Wilson, the founder of Software Carpentry, Jason Williams, who served on the Executive Council of Software Carpentry, Kate Hurtwick, who's on the current Executive Council, and Martha Graham on the Data Carpentry Executive Council. So this is really a community effort, and so I want to attribute that and represent that in this talk because it's a part of that inclusive approach that we're trying to take.
So I think we've seen throughout this meeting something that's really important is that we now have data and tools to advance progress and address questions in science and society. There's so much potential in what everybody's been talking about in the last couple of days with the data. And then just, I mean, the revolution in the tools available in the art ecosystem has been amazing over the last couple of years. We're in a place where we weren't even a couple years ago.
And we still talk a little bit, though, about the compute and the data. How do we bring compute to data or data to compute? But we need to talk about how we scale our impact, talking about bringing people to data.
So you're in this room. You could be at any other talk. You could be doing anything else with your life. Why is this the most important thing to be talking about? Why is teaching people how to work with data? We have a lot of social challenges. Why is this one the most important?
And so I want to point to this quote from Yashmi Milner, who's the founder of Data for Black Lives. And that data becomes a tool of profound social change or a weapon of political warfare depending on whose hands it is in. So we need to think about, as we have this data, that this data is power. And that we need to democratize these skills to give people the power of thinking about where they're headed, of how to interpret data, how to use data to answer the questions that are important for them.
data becomes a tool of profound social change or a weapon of political warfare depending on whose hands it is in.
So this, to me, is one of the most important questions that we're facing right now as we look ahead to where we're going and seeing a data-driven world ahead of us, in fact, that we're in already.
About the Carpentries
So then if this is the most important question, getting more people to be able to work with data, how do we scale the number of people who can do this? So that's what we're trying to think about in the Carpentries. So the Carpentries is a nonprofit organization, and we're an open global community teaching researchers the skills to turn data into knowledge. And we do this through curriculum, open collaborative curriculum, an instructor training, teaching people how to teach, and community by building and fostering a community around these practices and about sharing knowledge.
So I'm going to try an analog survey approach here. We'll see how it goes. So in reflecting on your own learning experiences, who in this room has been, and I'm going to tell you a way to answer this question, who in this room has been in a course or a workshop where they felt like they didn't belong? So if you say yes to that, we're going to clap twice, OK?
OK, so that's a couple of people. Good analog survey approach. OK, so how many people after having that experience left that topic for a while? So we have no idea how many people that is, but at least it's a few, right? So feeling like you belong in that course is really important for how you continue to learn that topic.
So in the carpentries, I'll talk about how many workshops we've taught, but it's been a lot. And so something that we realized when we were trying to measure our impact or think about what was important is that we realized it's not about a specific skill set. It's not about, can you parse this line of code? Can you construct a function in this way? But that it's about establishing a growth mindset around data and code and people feeling like they belong, that data science is for them.
And so the growth mindset is a bit of a loaded term right now, and does it work? Does it not work? So I'm not going to talk about that, but what I do want to say is, what does it mean to us right now when we say that? So it means confidence, the confidence that this is something you can do or continue doing, that we focus on effort and incremental learning and not outcomes, that you don't need to immediately get something right. And seeing mistakes as opportunities for learning and having an opportunity for a safe space to make those mistakes to continue learning.
Inclusive pedagogy: the four A's
So we want to work towards this growth mindset using inclusive pedagogy. So not focusing, we do focus on what we teach, that is a big thing, and I'm not talking about that right now, but how we teach. And so that's what I'm going to talk about now. And so we're thinking about it in the context, and for the people that are in the room that are saying, I've never actually heard us say it this way, that is true. So we're thinking about taking all of the things that we do every day and stepping back and saying, what is it that this represents? It's these four A's, accessible, approachable, aligned, and active.
So what do these things mean in the context of a learning environment? Accessible. So available in a time and a place where people can attend. So that's really important, right, because if you have things on weekends and you can't have childcare on weekends, you can't go on weekends, but maybe you can't get away from work during the week, or maybe you only can do evenings, maybe you only can do online because you can't leave your house. There are a lot of different ways that we can deliver things so that it's accessible to people in a time and a place that works for them.
That it's welcoming, so that when they are there, that it is accessible to them because of the environment that's created. So in all of our workshops, we have a code of conduct, and we have a reporting structure. Those are both important, not just having a code of conduct. And we start with that, I appreciate in this meeting, right, that they started with a code of conduct, because it sets the stage as saying that this is a place that is supposed to be accessible and welcoming to everybody.
And being proactive about ensuring accessibility needs. So I think we still have work to do in this front, but looking at your website resources, looking at your color schemes, a lot of great R packages around that. Looking at, through the lens of accessibility, for everybody, not just for the largest percentage of learners.
Approachable. So, great, this is accessible, I can go to this thing, but do I want to go? Is this for me? And so that starts at the very beginning, in advertising and recruitment for the workshop. If this is for beginners, say this is for people with little to no prior experience. In this domain, map out, essentially, what your persona is of this person who can come, and tell these people who it's for and why they're welcome. Invite them to participate.
Meeting learners where they are. So understanding that someone might be working in Excel, and that they maybe have, you know, primarily only had experience with their computers for Netflix. That's not a bad thing, that's where people are. Let's appreciate where people are and meet them there, rather than hoping that they're someplace else. And that is what helps make it approachable. We're creating that on-ramp for them, we're putting the on-ramp in front of them, instead of way the heck over there, or a humongous step in front of them to start. And that helps overcome the activation barriers to getting started.
And then, of course, the mistakes are the pedagogy. So in our workshops, we'll talk about how we use a participatory live coding approach. So you always make mistakes when you're doing it in person, but we treat that as a part of the learning opportunity. Seeing other people, when they make mistakes, when they don't know what's going on, helps you understand, one, how you approach fixing a mistake, and also that that's okay. And I think, since we're supposed to quote Hadley, that's something that Hadley always says, is, right, that, you know, he has to Google things, right? It's not that everyone knows everything and never makes mistakes.
Aligned. So they're all sort of similar, but the aligned is really about the motivation. So teaching with datasets that are relevant to the user. So we had a great discussion this morning at the Data for Good BoF about using datasets that the students were really excited about, and that got them excited about the topic. Because if you're trying to introduce some topic that they're not excited about and teach the skills on top of that, you're trying to do two things at the same time. So starting with a topic that people are excited about.
Teaching the skills and perspectives relevant to their current work. So yeah, you might hope that, you know, down the road, think that everybody should know C++, but if that person is not going to use it right away in their context, then it's not going to be something that sticks, that understands, they understand why it's important, and will motivate them to keep using the tool. Instructors that are current in the field. So peer instruction. So some of our best instructors are people who were learners only a year ago. Because when they understand the misconceptions, they're very familiar with what that process looks like. So when they see that someone doesn't understand something, then they can relate to that, help guide someone through that process. They also are peers. They're an ecologist or a social scientist or an economist. And so they can put the things into context.
And finally, active. And this is one of the ones that we really like to talk about. And this is a little bit that idea of guided practice, that you can get feedback on what you're doing as you're learning. That's one of the challenges of learning on your own, is that you often can't get that feedback right away.
And so participatory live coding, that's the instructor at the front of the room going through code and having the learners go through it at the same time. So you're participating together in doing that coding. An I, we, you model for teaching. So I describe something, we do it together, you do it yourself. It gives you a chance to try it out on your own, but not completely without a safety net.
All of our curriculum is open source and collaboratively developed and updated to represent the current practices. So our curriculum is active, actually, in and of itself. Oh, I see, I, Kara Wu, in fact, introduced tidyverse to our data carpentry lessons. So we have people who are in the field that are active and engaged in the topics so that they kind of meet those first three A's.
We have helpers in our workshops, so not only the instructor at the front of the room, but someone who goes around and talks with people to create a strong five to one student ratio. And then finally, our most famous probably is formative assessment through using sticky notes. So if you completed an exercise, you put a green sticky note on your laptop, and if you're having trouble, you put a pink sticky note, and someone will come help you. So it's a way during the workshop, not just waiting until the end and saying, oh, my gosh, did people get it? of seeing as you go, getting a sense of the room, assigning help when it's needed. So it's a formative assessment, assessing as you're going during the workshop.
Workshop outcomes and impact
So that was a bunch, like our whole concepts of instructor training. That is two days smushed into about five minutes. So you saw some of the instructor training earlier this week if you were there. But all of our instructor training curriculum is also online. Like all of our lessons, it's all CC by.
So the question is, if what we're aiming for is confidence and continued learning, how are we doing? How does this approach work in terms of meeting those identified goals? So first, how many workshops have we taught? So since 2012, we have over 1,600 instructors, over 39,000 learners, and we've taught over 1,700 workshops.
So not only have we taught a lot of them, we've had a global reach. So this is a map of the workshops over time, getting brighter in color as we have more workshops. And Francois did this great plot, and I'm super excited about it. And you can see we got Antarctica in the last year. So very excited about that. One person taught that. We think we should just put her name on Antarctica. She just claims the whole thing.
So we've also been able to have, because of this volunteer approach, approaching teaching from the perspective of meeting learners where they are and scaling impact, we've been able to reach a lot of different communities.
So the first one we're trying to achieve is confidence. So how do we do? So this is a survey of students. This is just data carpentry. It's not all R. Most of data carpentries are R, but some are Python as well. So pre and post, two days. How much has people's perception of their abilities and their confidence changed? So the first question is I'm able to write a program that would solve small problems. So we see these full point differences around write programs. I'm able to search for answers. I'm able to make my programming more reproducible, overcome a problem, become more efficient in programming. I know to keep my raw data raw, I'm very invested in that question. And then that last one, confidence in programming. So I have confidence in my ability to program. That was one of our lowest scores right before the workshop, and it's increasing a full point in two days.
You might say maybe that confidence is misplaced after two days. But the point is that they feel like it's for them. People feel like it's for them, right? And these are some of the words that people use to describe the workshop that reflect those ideals that we have around inclusivity and accessibility. And you always see instructors and helpful. So it represents kind of a lot of those things that I've talked about with our practices.
Not only are people more confident after two days, but when we survey people more than six months after a workshop, and this does have a bias because only certain people answer surveys that you send out way long later, what is the percentage? Basically almost 100% of people more than six months after a workshop say that they are more confident. I don't know. You don't look surprised. I'm surprised. I do programming. I don't always feel more confident after I've spent a day programming, I can tell you. So this to me is not only this sort of short-term effect of two days, but the long-term this feeling persists is very important.
So this to me is not only this sort of short-term effect of two days, but the long-term this feeling persists is very important.
Continued learning. Do people go on to continue learning? So this is, again, that longer-term survey. Have you gone on to continue learning after the workshop? So most people agree or strongly agree that they've gone on to continue learning. And these are using these skills in their work. Improving overall efficiency, managing data. So people are regularly using these practices. Again, when they come back to reflect on what this has meant to them, what they're capable of now, these are some of the words that they use.
And they strongly recommend their workshops to others. So almost 80% have recommended the workshop to someone else. Our net promoter score, if you're into net promoter scores, is like over nine. Is that the right thing? The number of people that want to recommend these to other people also reflects on how they felt about that experience, not just about what they thought they learned.
So safe spaces for continued learning. That was that last one around a goat's mindset, because we want people to continue learning. How do they do that? So one way that they're seeing that, people continue to engage with our community, sometimes going on to become instructors or helpers. But they do a lot of other things. Sometimes they use carpentry lessons, they use short courses, they go to meetups, they go to semester-long courses. So when we talk about this after the workshop, we do some of that. But what they're really doing is they're diving into a community of practice. And so seeing these practices that exist is really important. And so we're lucky here with Our Ladies and all the things that people talked about, that they do have welcoming communities of practice to continue to learn in.
And so then the question is, so great, awesome. People like it. But I said at the beginning that one of our goals is reaching diverse communities. And how are we doing? Well, gender, I guess, would say pretty rapid. This is just in the United States, because that's the only place we collect demographic data because of legal reasons. We're not doing so well. So we need to do better.
So there are elements of creating a space where people feel welcome. There's more opportunities for that. There's more opportunities for recruitment. And so what we want to say here is that with this ecosystem, with this community of practice that people are building, with all of these different teaching practices that we together are an ecosystem. And that if we want to go fast, we can go alone. And then if we want to go far, we go together. And there's great opportunities for that here. And we look forward to continuing to work with all of these communities and building our future together around data. Thank you.
