Panel Q&A: Real World Applications of Open Source in Public Health
Please join us for the live Q&A session at ~11:25 am ET for the July 17th Event: Real World Applications of Open Source in Public Health This Q&A Room will open up immediately following the presentation. For anonymous questions, you can use: https://pos.it/demo-questions Speakers from the following organizations will join to discuss how their team's work has evolved after learning R or Python through Posit Academy: Idaho Southwest Health District Oregon Health Authority Vermont Department of Health What is Posit Academy? Posit Academy is a mentor-led apprentice program that follows a project-based learning format. Like riding the bike or playing the piano, Posit Academy participants learn by doing with real-world applications. Participants are placed in small groups, under the guidance of a mentor, to motivate and support one another throughout their learning journey. If you'd like to learn more about Posit Academy, you can book a meeting with the Posit team here: https://posit.co/talk-to-us-about-academy/
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
All right, everyone. Welcome back. Thank you for watching our presentations today. Hopefully, you got some insights from our customer panelists. They'll be joining us right now. So, as they enter the room, we'll take a moment. We'll go around and introduce everyone here that you see, and then we'll get started taking your questions. So, I'll lead us off. My name is Lindsay. I work at Posit full-time, and I dedicated to Posit Academy, our training program. So, for the past two years, I've been helping our customers learn how to do code-first data science. I'll hand it over to Kristen. Hi, folks. My name is Kristen Batt. I'm part of the Posit Academy team, and I spend a lot of time working with learners, mentors, customers, trying to figure out the best way to connect people with the knowledge they need to be more effective at their jobs. So, that's me. Glenn, over to you. Hey, everybody. My name is Glenn Klausner. I'm an account manager here at Posit.
I've been here for a few years now working on the public sector team. I've been working with all these great folks for the last couple of years, and I'm excited that they had the time and energy to share some of their experiences. Andy? Thanks, Glenn. Hi, I'm Andy Nunez-Brewster. I'm with Oregon Health Authority, data equity analyst, and just really happy to be here and glad to have this opportunity to connect with folks.
Ricky? Okay. Hello, everyone. My name is Ricky Bowman. I'm the program manager for our public health emergency preparedness and epidemiology response program here at Southwest District Health in Caldwell, Idaho. We're about 20 miles west of Boise, and we're a local public health department that serves six of our southwest Idaho rural counties.
Thanks, Ricky. Maybe we'll stay in Idaho and pass it over to Lakshmi.
Thanks, Lakshmi. Jessie? Hi, I'm Jessie Hammonds with the Vermont Department of Health. I'm the division director for the Division of Health Statistics and Informatics. Nice to be here with you all today. And Amanda? Hello, everyone. My name is Amanda Jones, and I also work at the lovely Vermont Department of Health, which is the state public health agency in Vermont, and I am the informatics and data modernization director. Awesome.
Giving teams grace during change
Thanks, everyone. All right. Let's get started with your questions. So over at the link for Q&A, yeah, so you can drop your questions in the Slido, you can upvote them, and we'll take them here. So the first one is for Vermont. You said change is hard and talked about giving people grace. What did you do as leaders to give your team members that grace?
I think there's multiple ways to answer that question, and Jessie will take one, which one very big piece is communication. But I want to take people's attention to the word leaders. We may be leaders now, but we worked at the Department of Health for a long time as analysts. So I feel like when we're talking about that we gave people grace, it's because we're coming from a place of understanding. We have learned new things before related to coding and software. So I think it's not lip service when it comes from someone that actually has been in a similar position. So I think that was one of the helpful ways of giving people grace is actually being able to be like, yeah, I know I've done this, too. It's hard, and learning stuff as an adult is even more hard when you're in a career and you have 40 hours of work anyway. So I think the way in which the empathy, obviously, but also being people that actually speak that language was probably also very helpful.
And as Amanda said, I wanted to speak to communication. So one of the ways was a lot of the staff who are doing this work have gone through the training that we worked in closet on. We got buy-in from our programmatic partners up front. They all understood and agreed with the staff that worked with them participating. And we had discussions about what that would lead to and that there might be some short-term impacts related to efficiency as people started making those changes and kind of had to redo some things or rethink some things. So they knew that up front. And I would say as we went on, if something comes up, as leaders, we handle the front of those communications that helps give the people doing the work the space so that it takes some of that pressure off of them. And then also as a part of what Amanda was saying, we communicate with the staff and told them we know that in the short term this might impact your ability to do work quickly and that's okay. And that we just need to communicate to me when the impact is so that we can make sure everyone's on board and understand that. But just really acknowledging with them that it's okay.
Automation and time savings
Awesome. Next question. Andy, you mentioned automation. How much time has your team saved?
Hi, Glenn. Thanks for that question. I think we're still trying to quantify that. But I can just say from my experience that the initial upfront investment of building the code, it saves so much time in going back and running that report again or adapting a report instead of doing the whole thing from scratch all over again if you have one little change. So just ballpark figure, I mean, probably half of the time that we used to spend just doing manual work is now freed up to do other things. And so we're investing some of that time back into learning and we're investing some of that time into developing new workflows.
What data modernization means
Thank you, Andy. So a question I'll actually kick to everybody. So data modernization has come up a few times. Curious to hear from all of the groups if you could speak about what that means to your organization. And we will take this in alphabetical order. So that being Idaho, Oregon. So Lakshmi and Vicky, over to you.
So data modernization is how seamlessly we can integrate with systems being a public health district and working on the communicable diseases jurisdiction. What's important to us is to gather the data, the quality of the data, time receiving this data. So for us, the important aspect is to integrate with the hospitals, the electronic health systems, and how to integrate this into our surveillance systems so that we get these disease notifications and are able to identify real time triggers of infectious disease outbreaks and concerning trends like that. So basically for us, it's integration and timeliness.
Yeah, Kristin, and I can add a little bit to what Lakshmi says. Just on my end, what data modernization means to the team and to the organization really is trust. One of the things that we have really tried to do with our data modernization using the R platform is build that transparency with our local residents so that they can know of what disease trends are out there, what outbreaks that we're seeing, and then ultimately what we're doing by using these tools is really building trust within our local communities that they can trust us knowing that we're going to be able to provide the best data for them in a way that they can make the best decisions for themselves and for their families.
Fantastic. Thank you. Andy, how about you? Yeah, I definitely feel like we echo some of that similar sentiment around transparency and trust. And in Oregon, there's also a significant amount of grant money from the Centers for Disease Control and their data modernization initiative. And so we actually have some fiscal responsibilities directly related to data modernization. In addition to Oregon has an overall goal of eliminating health inequities in Oregon by 2030, and that's a very ambitious goal. And so our modernization and our equity goals are intertwined and are being worked in parallel so that as we modernize, as we move data systems to the cloud, make them more accessible and more usable, we are also looking at how we can leverage that effort to directly eliminate those inequities to bring data back to communities. And again, I think that comes full circle back to trust and transparency.
Fantastic. And Vermont folks, perspective from your side.
It's my job title. So I feel very strongly about those two words, which yes, is a CDC, national public health initiative. But what it means to me, and there's a lot, depending on a different jurisdiction, people have different answers for this. But for me personally, it really is like, how does public health work with data in their systems? And we need to make that better. And I think anyone can agree that worked on the COVID response, that our systems were not ready to deal with that and get data out to make good public health decisions. So how we view data modernization is we have like a million projects and a million layers, but related to this is really equipping our workforce. Because what is the point of updating a system to get a bunch of data that we can't use in an equitable way, an accessible way? So it's really about for this related to workforce and equipping our workforce with more skills to be able to work with data in a more interesting, creative way to give data to our programmatic partners and community partners to make better decisions for better public health outcomes.
What is the point of updating a system to get a bunch of data that we can't use in an equitable way, an accessible way?
Time to results: from outsourcing to in-house
I'm going to go off script here for a second, because something you all just said made me think of a prior conversation I had with Ricky talking about prompt results to serve the community. Ricky, I know in an earlier conversation we talked about how before learning to code, you all maybe outsourced some reporting and data analysis that maybe took you a bit longer. Then after you learned to code, you might have been able to do that faster. Could you maybe give a brief recap of that, kind of the time to results and how Posit Academy helped you with that?
Yeah, absolutely, Lindsay, and thank you for that question. So during our COVID response, we didn't have any data visualization capability with our organization. So a lot of the requests from whether it's our elected officials or even community residents themselves was to have some kind of data visualization on our website that tracked the incidence rates of what was happening. With that being said, we needed to outsource that to a local contractor that we had and just the response times and for us wanting to keep up with the demands of the data was just not there, Lindsay. So we were looking anywhere from eight, nine days of having that outsourced into once we have the ability to ourselves do that within minutes and hours. So that has really just saved us tremendously on the time being able to do that. And then going back to my original comment with the trust and transparency, it just builds more trust in our community knowing that we can be able to respond in a timely way.
We were looking anywhere from eight, nine days of having that outsourced into once we have the ability to ourselves do that within minutes and hours.
Persuading others to learn to code
I'm the only person using R on my team. Any advice for persuading other people to learn to code?
Anyone want to take it? Andy, I know you've done a lot of work with the community in Oregon.
Yeah, this is the thing that we hear occasionally and a lot of this is just that our teams are kind of siloed. There's a lot of effort to have more collaboration and more sharing of resources and project templates and things like that. So the way that we've approached this at OHA is creating that larger community of practice so that the one R user from any team can connect with other folks that are the only R user on their program and their team and things like that. So we've seen a lot of engagement that way. And the other thing I guess I would offer is if you're in a smaller agency and there just aren't that many other teams with one other R user on them to do that would be to connect with your local R user group meetup or R ladies is just about everywhere and they welcome everyone. So finding those affinity spaces I think is really important.
We certainly have done work connecting teams where there's fewer users with other teams where they're making more. But I think also we've found that R users are really have been fairly passionate or pretty passionate about using the software. And so some of them have one person in particular kind of tackled putting together some crosswalks between R and whatever the other platform is so that it made it a little bit easier for some people to understand how they can do their work in R as well. So that has been really helpful and several other people on that team have now started using it in some way.
Lowering the barrier of entry for non-technical learners
Can you speak about how Posit Academy lowers the barrier of entry for those with a non-technical background?
I will say the people that did the academy so far are classified public health analysts, so they probably would not be considered that. However, the academy that we'll be having in the fall, we actually opened up to other divisions because we're like the centralized nerd division. So everyone's an analyst, everyone knows how to code. But we actually expanded it to include people in other divisions with a data role, but they're not necessarily like data analysts. They may just be a little bit more familiar with data, which, you know, I think people like a challenge. And I think as long as you frame it in an encouraging way, you know, and there's mentors, and the nice thing is, is that we have people that have gone through it. So we will have an internal support system as well.
Yeah, so one of the fun things about my role within the academy is I get to talk to a bunch of different customers, a bunch of different learners, and sort of hear how things are resonating with them. So the things that we've heard this morning, so Andy talked about community of practice, Amanda just mentioned like the importance of having an internal community. I think that's part of it. And one of the things that I think is special about academy as a program is that we have that community focus. You have the mentor, you have your other colleagues that are learners in the group, and you can turn to each other and be like, oh, man, like, Lindsay, this is hard. Like, I'm new to this. This is like, this is scary. And it feels kind of like sticky and big and ah. And I know having seen these groups work through academy, this does, that social component is really important. I think having the mentor there, having your colleagues, and the material is written in a way that it is pitched at people who don't know, we do not expect you to come in and understand what programming is or what a function does. So I think there's a bunch of different ways we're trying to make this a space where people can come in wherever they are and get up to speed. The last thing I will say before I pass back to you, Glenn, is I think that starting out with the why are we doing this, the kinds of things that we've been hearing in this room. So, you know, like, why does this matter? This is going to take me a long time to learn. Like, yes, but when you get there, you're going to be able to, like Ricky said, own your own reports. You're going to be able to do things more quickly. You're going to be able to do things more reproducibly, which means cross team communication is easier. People changing jobs is easier. Just general, the resiliency of your whole organization goes up, and your ability to produce more efficiently comes up. So I think with that as a starting point, the community to add on, oh, okay, no, I can do this. And then great things follow.
for our academy teams, everyone was an analyst, like in Vermont. However, one thing I think that it did lower the barriers to entry on is learning the new, Kristen, you mentioned the social aspects of it, but there are social processes and skills that we're learning as we're learning our coding skills, and the ability to practice those as you're doing the technical learning, I think is a really, really key piece, because we found that the folks who did the academy came away with more confidence in being able to do code reviews, for example, or just being able to know when to ask for help and not feel bad about that, right? Like huge, huge things. So for both technical people and for non-technical people, I think the social, the practice around social skill building and getting comfortable with not knowing something and figuring it out, I think is huge.
Stakeholder feedback on reports
I'm curious, you know, especially Ricky, Jesse, what's been the feedback people are receiving about the reports? You know, we've heard from many that leadership is used to receiving Excel. Do you have to change the minds of stakeholders to execute on this?
The feedback we've received has been positive. People like the look of the reports. We've been able to build in our department brand standards, so visually they're pleasing and look similar to other reports, and yeah, no, I'm not aware of any negative feedback that we've received. In fact, we do do a lot of like stakeholder feedback during development as well, so like we will show people drafts of things to give feedback to, so you don't want to show leadership something fresh that they've never seen before, so that's also helpful.
Yeah, and for us, we've been able to seamlessly implement our reports into our weekly report to leadership, and we haven't had, like Jesse mentioned, any kind of negative or criticisms on the look and feel of anything. It's just been positive. One of the things that I can say is it's been more intuitive for our senior leadership to learn and follow and to ask more questions because we're more curious about just the look and feel of what we're showing and demonstrating, but also, too, what I have noticed is that with the reports that we do specifically, that they're a lot more engaging than, let's say, just a standard bar graph from like an Excel document or a scatter plot from an Excel document. It really pops out, and it brings the viewers in to want to learn more and ask more questions, so it's nothing but positive on our end.
I'd also like to add to what Ricky said. I'm sure that my audio was not working, but I hope it's fixed now. Okay, so the point is that it's free to use. You don't have to pay. We started using Excel hours over hours to create an opportunity to include more statistical models into your data. It's always important to know the data, so that we identify trigger points earlier, which is also helpful for the stakeholders and the leadership to kind of trigger them on what's going on early, so we can implement interventions at the earliest as well. So that is very important in public health. So beyond using Excel, R opens up a great realm of opportunities for statistical analysis and also data visualization.
Lasting impacts and continuing to code post-academy
curious for anybody in the room about the lasting impacts of academies, so do you see people continuing to use code post-academy, or, you know, was it sort of like, okay, I did this thing, that was cute, and now I'm going to go back to living in my Excel living room and clicking on a bunch of things. So I'm going to toss this over to Andy first, because he's next on my screen.
Thanks, Kirsten. Yeah, I want to, there was a question that flashed up here right before this one about the move from BI tools, and so I'm going to thread that one into this here, because that is a previous, you know, workflow that some people were used to, and then they learned how to code, and now they're doing something, a slightly different kind of a workflow, and what we have found so far with Academy is that folks are continuing to invest in their code-based workflows. In fact, there's been sort of this, like, not really like a ripple effect, but like a sharing of, look at what I've been able to do with this, and then other folks want to kind of see that and learn from that, and so I think what it's doing is it's kind of shifting into this more open opportunity space, where we can kind of get curious about each other's work, and I'm not really hearing much in the way of folks, like, wanting to go back to the way things were before, because a lot of, there's a lot of pain points that were addressed in this, even though there's an upfront investment of learning to do the code and then developing the code, I think it's really addressed a lot of, you know, old pain points.
Vermont and or Ricky, just because Lakshmi, we're having a hard time with your mic still. Any thoughts from your side on, you know, changes in workflow, things like BI, or folks continuing to code?
I would echo largely what Andy said. Staff are continuing to code, continuing to work on shifting their workflows, looking for more opportunities to use it. In terms of the BI question, I think there will be more shifting in that, away from traditional opportunities. It's raised a lot of questions for us about decision-making as to which tool you should use when. So, we're working through some of those conversations now, but for sure, the impact is felt and is continuing to grow.
Is Posit Academy geared toward public health data?
Okay. Glad to hear it. Keeping an eye on the clock. Lindsay, back to you, if you want to wrap us up. Sure. And I, with looking at the time, I think this will be our last question. Is Posit Academy specifically geared toward public health data?
This might be a question here for the Academy team. So, we, not entirely. We serve customers across multiple industries. For each of these groups, the data sets they worked with was relevant to public health. Andy, or Vermont, or Ricky, anyone you want to chime in on maybe what the data was like that you were working with during the learning experience?
Sure. I can take this one. So, data were in a, just in a CSV file. So, in a format that we're all familiar with working with, and formatted kind of similar to what you would receive if you were pulling data from like a, maybe a summary from like a patient care data system, right? So, you know, a handful of outcomes and treatments that your population had, and then your practices building a set of either functions, or skill sets, or routines, or processes around doing different things with this kind of same set of data. We actually really appreciated that the data was there was some continuity between the different Academy cohorts that we had. And so, we really appreciated that aspect of it.
And Lakshmi's continuing to have audio issues. So, I'm just going to grab some things she said in our internal chat. So, the report from the Idaho folks is that the data is very clear, geared to what they work with in public health, and it looked very similar to familiar data. So, from the Academy side, I will say that we do adjust the focal data science project data set to wherever the learners are coming from. And we do that because it makes the learning more relevant. It's very motivating for people. It helps make the material click a little bit more. You can say, oh, neat, not only like I'm learning these skills, but I'm learning these skills in a context that makes sense to me and resonates.
Thanks, everyone. For the questions we weren't able to get to, we'll write up a blog post, something to post publicly so we can get you all those answers. And I'll pass it over to Glenn for a farewell message. Thank you all for joining. Yeah. Thanks, Lindsay. I really just wanted to say thank you to everyone, right? Most of all the panelists. We really appreciate your time and the investment that you've made into working with us to share the message. As you know, we're incredibly excited about Posit Academy and the breadth of capabilities that it brings to the market and the excitement that we have as it relates to the teams and the feedback that we're receiving. So, again, thank you for participating. For everyone who participated and watched this, we hope you enjoyed it. I think you hopefully learned a lot of great information from an incredible group of folks. I suspect that any one of them would be more than happy to chat with you. I don't want to open their doors up too much, but I know for Lindsay, Kristen, and I, we'd be more than happy to chat with you about opportunities to learn more about Posit Academy and or Posit Technologies. So, please feel free to hit us up here at Posit, and we will look forward to speaking again with you soon. Thanks again, everybody. Hope you all have a great day.