Posit Meetup | Ryan Derickson, VA | Collecting & Acting on Employee Engagement with Shiny

There are a lot more people who can say no than can say yes. So you need to find that person who can really motivate people to work on this project on your behalf.

Thank you. I'll save some of the other questions for a little bit later. So I'll let you get back to the talk. Thank you. I did see one that's easy to take. Limitations on VHA's Azure Space CRAN only or GitHub. Haven't run into any limitations yet. We try to stay with CRAN just for reliability reasons, but no issues there.

Engagement, burnout, and DE&I data

So next, we talk about employee engagement and burnout, and we do that in the same way and with the same kind of interface for both of those constructs, just, again, for kind of predictability reasons. So we talk about engagement in terms of the percent of the organization that's engaged versus mixed or kind of on the fence, you know, maybe engaged sometimes but not always versus disengaged. So the percents are easy to see here as a bar graph, and then there's always a change score toggle that's a plus minus.

The VA's dots are for reference again. So if you're the director of Boston or whatever facility, you can easily see where you fall against VA itself. And then for every bar graph, the one layer down here is you can click on the bar and the little cursor kind of turns into a hand to kind of cue you into that. You can click on the bar and see a drill down in a few different ways on that specific construct. So for disengagement here, for example, we see it by supervisory level from most disengaged to least.

By service, a lot of variance here. Again, fake data, the little banner at the top gives that away, but you do see, you know, 40, 50, sometimes more points of variance at the service level. Work group, again, those small teams of 5, 10, 15 people. And you see the most variance here because people who are in the same occupation don't all have the same supervisor. And so that's why there's usually more variance at the work group level. There's a supervisor effect, basically. And then info will just have background on that specific construct.

So this is nice because we've got, you know, maybe 15, 20 people doing these briefings, and some people are more or less familiar with some of the details of the measures. So it's a great reference for us, but also for our executive audience because we leave this app up all year, and there's a lot of traffic throughout the year. People come back to it. And so you can easily remind yourself what the details of that measure are.

For engagement and for burnout, we also have hotspot and bright spot maps. So these are like those maps you saw a minute ago, but now we pull specific groups out that have either a concerning score, in the case of a hotspot, or a good score in the case of a bright spot. So these are groups where disengagement is at least 50%. So you get that quick kind of triage, and again, capitalizing on the schema of the org chart that people are already familiar with. Same thing for bright spots, engagement of 75%. So in this case, for example, it's useful to see the org chart display because if you're the chief of social work, you can see you've got three groups that meet this threshold.

So burnout is next, and it's the same kind of display for predictability reasons. Here we talk about the percent of the organization that has zero, one, two, or three out of three symptoms of burnout at least once a week. So the more symptoms you have, of course, the more burned out you feel. So the bar graphs show the scores, the dots for reference for VA overall, the drill downs by clicking on the bar.

We have free text themes here as well. So in the survey that this data comes from, people can also leave free text comments, and we theme those and report them out verbatim as well. We bring in the themes here just as a kind of an idea of what's top of mind for people when they're prompted to say something about their workplace. So as you can probably imagine in healthcare, especially now, burnout is top of mind for a lot of folks. So you see that quantitatively and also qualitatively in the comments.

And then we have two questions, a lot of bright spots here in this fake set. We have two comments about COVID, or two questions about COVID rather. Extra professional stress because of COVID and extra personal stress because of COVID. And the scale is from none to extreme, so you can see the distribution there. And then I'll spare you the detail again, but you can click on the bar and see the same drill downs as you've seen so far.

Any questions before I jump to the next section? Yes, there are a few other questions coming in from YouTube and LinkedIn. One was, can you touch upon performance, giving multiple users using the app simultaneously? So we have a, it's a 16 core, 64 gig server, and we can handle about 100 people at the same time. That's across a few applications though, and some are more RAM heavy than this is, although this is still pretty heavy. There are huge data sets that go into this and we try to cut them down and do as much pre-work as we can, but there's no way around some of it. So we do okay with 100 at that size server.

You can play with the settings and connect how eager versus unwilling it is to spin up a new session for incoming users. So you can have more people piled onto the same process or you can have more processes. It's some experimentation. There's also shiny cannon. I think that's the search term you want to use to do some testing, but that's at least our experience.

Another question that was over on Slido that I see has been upvoted a few times was, was the UI all done with bslib or did you also incorporate JavaScript and CSS? It's, the base is bslib and there's a lot of custom CSS. Yeah. Not so much JavaScript. I think the only script I can think of offhand is what toggles it to be full screen to remove the browser noise at the top. So not a lot of JavaScript, but a lot of CSS.

And this question I think was shared in a few different places, but it was, can you either share code with us or give general guidance on where to learn the click on like drill down interface function? Yeah, the, so the, the maps, I'll start at the top. The maps are from, let me go back up here. These maps are using collapsible tree. It's like, that's the package name. So there's a, there's a shiny binding that gives you the input. So you can use that to generate a modal that has the, the information for the thing that was clicked on. So that's how to do it here.

For bar graphs, these are all done with Plotly and there's similarly, there's a, there's a binding for shiny, there's a click binding that you can use to reference the, for example, the row number of the dataset that corresponds to the bar that was clicked on. So it's a little finicky to set up, but once you get it, it's pretty reliable. That's, that's how we do it.

I see somebody had asked if the session will be available later. And yes, right after the session, the recording goes up at the same YouTube live link.

How could I integrate an R Markdown file which could export more than one interactive plot at once in a shiny application? I don't know. Haven't done it. Best of luck though. We can, we could try and find some other resources too, or if anybody has ideas, feel free to share them in the chat.

And then one other, sorry, one other question. Eugene asked to clarify, how do you serve the dashboard? Is it on RStudio Connect or? Yeah, we, so we have a Connect instance that's on Azure. And so we just get a link for this page and send it out to everybody.

I see Matthew had a question too about, it's anonymous, but it can be attributed to the manager. Yeah. So the intent of this, and this takes a lot of work and sort of institutional familiarity to actually make it stick, but we've been doing this for, you know, 15 years or so now. So we've got some of that. The purpose of this is purely developmental. The only thing that's in performance plans has to do with using the data, not about getting a specific score or anything like that. So yeah, by mapping it on the org chart, there is like, you can see who supervises what group, but virtually no one uses it for that purpose because the data wouldn't be useful then if we did that. Like as soon as you create a standard that's going to be mandated or you create a bar that has to be cleared, then people find ways to clear it, even if that may not be how they actually feel.

Priorities, turnover, and just culture

So the next section is a different kind of display, just a table format here for DE&I data. The cool thing here, though, is the comparisons we do, not really the display itself. So we've got four DE&I questions, and those are linked above, but we use those and also three overall experience indicators. Best places to work, engagement, and burnout. And for every demographic, we test the minority against the majority and then shade where the minority is lower than the majority. So if you see a lot of red, basically, it means that in this hypothetical organization, people who you see have red cells are reporting more discrepant experiences on the topics you see the red cells for than the majority categories for their demographic.

So for disability status, for example, people who said they do have a disability, 21% discrimination versus 11% for people who say they don't have a disability. And there are two pages because we have so many demographics that would make too big of a table to show it at once, but as you can imagine, this is very interesting for executives to see. And there's also change scores, again, available here. And for those, the comparison is every group to itself last year rather than to its majority.

The next piece is about intersectionality because, of course, one demographic doesn't describe anybody very well. So here you can pick patterns of demographics and see how people feel about those same measures, but who meet multiple demographic conditions. Sorted so that it's high to low discrimination. And, of course, the coloring is a basic heat map scheme.

And finally, we have hotspots again here for those four questions. So for each of those questions, we can pull, again, those groups out of the org chart and see where some concerning scores may be. We don't do bright spots here because there's no way to know whether a group is being very equitable and inclusive and so on or whether it may just not be a diverse group in the first place.

Second to last data piece we have here, priorities and turnover intent. So priorities show what people in the group say they most want to work on over the next year. So we can see that prioritization, same click events as before, same thing for turnover intent, which, as you can imagine, if you're an executive, you want to know who's thinking of leaving and where you may have some staffing shortages, and then what's pushing people away. So the reasons people who said they want to leave gave for wanting to leave in the first place.

The last data piece here has to do with just culture. So if you're not familiar with that term, the idea is just that people need to feel, especially in safety-critical applications like healthcare, need to feel like they can speak up and say when something's not right, but also that there are standards and they're transparent and equitable and so on. So you want to be up here in this top right area, and you don't want to be in this off-diagonal. So this is a plotly scatterplot. The background colors, of course, just kind of cue you into where the desirable range of scores for these issues are. And you can change the questions to get different views. You can search and see, for example, all nursing groups at once if you wanted to. And then you can also toggle between work groups, which is the default, and between occupations, because, again, those are similar but specific.

And finally, there's a table below that has these same scores. But you can also, if you wanted to see the lowest groups all at once, you can drag a lasso, and the table updates to only show those groups, and then you can download this table easily to Excel. The tables are using the DT package. So you get the sortability and the downloading and so forth kind of by default, or at least very easily through that interface.

Executives usually want to recognize groups that do well, so it's easy to get additional resources we make available here or to see a table that's sorted automatically by default just to make it easy to do that. And then finally, we have just a bunch of extra resources that people have gotten used to seeing in VA, extra reports, the data stores, another app that's on Connect that we made that makes it really easy to pull specific data in dataset form versus in report form, other dashboards and tools and videos and so on.

The bookmark button is great if you're using Connect because, or I think Shiny Apps does it also. This will give you a custom URL

Posit Meetup | Ryan Derickson, VA | Collecting & Acting on Employee Engagement with Shiny

Transcript#

Introduction and VA context

The data: All-Employee Survey

Executives need to know how to help

Shiny app demo

Storytelling within the application

Q&A

Engagement, burnout, and DE&I data

Priorities, turnover, and just culture

Featured software#

bslib

Shiny