Building a Flexible, Scaleable Self-Serve Reporting System with Shiny - posit::conf(2023)
Presented by Natalie O'Shea Working in the high-touch world of consulting, our team needed to develop a reporting system that was flexible enough to be tailored to the specific needs of any given partner while still reducing the highly manual nature of populating client-ready slide decks with various metrics and data visualizations. Utilizing the extensive resources developed by the R user community, I was able to create a flexible, scalable reporting system that allows users to populate templated Google slide decks with metrics and professional-grade visualizations using data pulled directly from our database at the time of query. This streamlined approach enables our consultants to spend less time copy-pasting data from one channel to another and instead focus on what they do best: surfacing business-relevant insights and recommendations for our partners. By sharing my approach to customizable self-serve reporting in Shiny, I hope attendees will walk away with new ideas about how to combine parameterized reporting and dashboard development to get the best of both worlds. Additionally, I hope to end by sharing how this project was pivotal in making the business case for procuring Posit products for my broader organization. Presented at Posit Conference, between Sept 19-20 2023, Learn more at posit.co/conference. -------------------------- Talk Track: Bridging the gap between data scientists and decision makers. Session Code: TALK-1075
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hi, so I'm a little nervous. This is my first conference talk since before the pandemic, so it's very intimidating being up here in front of all of you, not just in my living room. But anyways, my name is Natalie O'Shea. I am an analytics consultant at BetterUp. We're a startup offering digital coaching experiences focused on helping people everywhere live their lives with greater clarity, purpose, and passion. My role on the People Insights team is to help develop and scale data-driven behavioral insights to support organizational transformation at some of the world's leading companies.
The reporting challenge
On our team, we're often dealing with scenarios like this. So we might partner with an organization that has made major publicly announced goals to advance equity on their engineering team, for example. After years of ineffective trainings and seminars, they might turn to BetterUp as a data and science-backed solution for implementing and measuring their admirable organizational transformation goals.
So in this example, they might have goals for the deployment where they want to, say, improve overall belonging for ERG leaders and underrepresented ICs while simultaneously improving inclusive leadership among managers. They might have a contract stipulation that says that we need to then map their BetterUp's whole person model to their unique cultural value competency framework. And then they want these results reported back to them at regular business reviews to track progress towards their organizational transformation goals.
So this is an account you're really excited to support, but there's one major problem. You're a CSM, and you're responsible for this amazing, inspiring account, which doesn't have the support of a dedicated behavioral science expert alongside 15 other amazing, inspiring accounts. And based on our internal estimates, it takes the full account team approximately 18 hours to build a single business review, with seven of those hours dedicated to just navigating different dashboards to collect the data insights needed to demonstrate the impact that coaching is having on a partner organization's transformation goals.
So rut-row, indeed. So this left our team with two crucial questions. The first being, how do we enable our field team to tell compelling stories of organizational transformation at scale with limited subject matter expert support? And once we've mastered that, how do we then productize those insights and enable our partners to have real-time insight into their own transformational goals?
Evaluating existing tools
So apologies for the very big table here, but you guys are all data scientists, we love tables. The current patchwork of analytic tools and approaches really fails to meet our needs for scalable, self-serve reporting in a consulting context. So in-product analytics often don't address the unique customer needs, and they often lack the ability to tell a compelling story in the language of the client. While on the other side of the spectrum, ad hoc customer reporting is time-consuming and difficult to scale effectively to meet customer demand sustainably.
Therefore, as companies, we often spend tons of time and money building self-serve analytic systems using business intelligence tools, but we're still left with this huge burden of effective enablement of less technical data consumers that often complain of information overload, and understandably so. So at the end of all of this, we still have often simple visualizations that lack visual polish, which is not something we're particularly proud to share with our clients.
One alternative code-based approach that I particularly love is parametrized reporting. This provides a much more customizable and scalable end product with professional-grade data visualizations. However, it often lacks the ability to then customize in those final stages and really tailor insights to the specific client goals and needs.
So ultimately, none of these approaches quite fit the mold of what we're looking for. Instead, we want to bridge the gap between software-as-a-service and standard analytics consulting with a tool that is simultaneously scalable and easy to use, while still enabling lots of downstream tweaking and customization. More specifically, we were looking for something that lives kind of squarely between business intelligence tools like Wooker and something like code-based parametrized reporting.
So through lots of user interviews, I learned that, surprise, surprise, no one really loves manually loading and taking screenshots from 10 different dashboards to build a single presentation. Instead, users wanted the convenience of self-serve tools, but also the visual polish and automation capabilities that are really only possible with more code-based approaches. So I said, por que no los dos? Let's do it.
Introducing BetterUp Insights
So introducing BetterUp Insights. BetterUp Insights is a Shiny app that I built to enable scalable, flexible, self-serve reporting that allows members of our field team to select a reporting group of interest, whether that's a specific partner or perhaps an entire industry, which is often useful for pre-sales motions, and then generate a Google slide deck populated with various data insights, which can then be further tweaked and modified downstream to help tell a really custom, tailored story of organizational transformation in the language of the client.
Just like any standard business intelligence tool, data is pulled directly from our database at the time the user clicks load data, but we go one step further by not only generating professional-grade visualizations, but then directly sending those visualizations and metrics to pre-formatted Google slide decks with several different templates, including things like inclusive leadership, coaching culture, hybrid work, and manager effectiveness.
This workflow is not only much more efficient and enjoyable, but it also greatly cuts down on opportunities for human error. So things like making sure that the date the data was pulled or end counts displayed underneath of a visualization in the caption, these may seem small, but they can make or break deals in an economic environment where every single business purchase is highly scrutinized.
deals in an economic environment where every single business purchase is highly scrutinized.
App build architecture
Of course, this wouldn't be PositConf if I didn't get to geek out a little bit about the build architecture of this app. So here comes the more technical stuff, but feel free to ping me after if you want to talk more.
First and foremost, I am an early and enthusiastic adopter of the Rhino framework for scalable and robust Shiny apps. So having developed several Frankenstein monolithic Shiny apps in the past myself, I knew I wanted to take a module-first approach. And while I've dabbled in other module-first frameworks like Gollum, I really prefer to keep my package development workflow separate from my app development workflow. So for me, I found the Rhino framework really intuitive and a joy to work with. So if you have a similar development mindset, I highly encourage you to check it out.
Next I knew that I needed to carefully manage reactivity within the app. So beyond just making some kind of standard design choices, like making sure that we only run queries when a user clicks an action button and not just reactively when anything changes, I also adopted the R6 plus gargoyle triggers approach, which has been advocated by Juwan Hill, which I link below. I found this especially helpful because I can then embed small helper functions within the R6 object itself that allow me to do things like apply minimum data thresholds for data privacy reasons in a very easy to maintain way.
And then finally, I was able to integrate much of my better app colleague Andrew Reese's previous Python workflows for working with the Google API into my Shiny app using the reticulate package, which I am fully convinced is pure magic.
Organizational impact
So what has this unlocked for our organization? The apps really become a crucial component of this emerging framework for efficient, unified, and field-tested partner reporting. So rather than just kind of willy-nilly building out product analytics, we really want to start with identifying and prioritizing that reporting need and then conducting analysis and visualizing data before we spend the time and effort really building that out into in-product data visualization tools.
So in addition to just the obvious efficiency gains of this approach, one additional benefit is more strategic role alignment for our field team members. So in my initial user interviews with members of our field team, one key theme that emerged in every single conversation that I had was an overwhelming desire to lean into the more consultative elements of a sales role. So users of the app wanted to spend less time copying and pasting data from one channel to another so they could instead focus on surfacing business-relevant insights and recommendations for our partners.
Making the case for Posit at your organization
So ultimately, BetterUp Insights was somewhat of a Trojan horse for getting our organization to adopt Posit products. And as I was telling some colleagues last night, I came into this role with admittedly a bit of an agenda. This was not my first time trying to advocate for Posit products to be procured at my company. But it was, in fact, my first time being successful in that endeavor. So I'd like to spend the remainder of my talk sharing some insights I gleaned through this process so that you might be able to take a similar approach for building the business case at Posit at your organization.
So perhaps the most important insight that I gained was that you need to identify a core capability that these products will unlock for your organization. So what capability can you uniquely provide that's not already present in existing tools? So figure that out, do it, and do it well.
So one pitfall that many of us, including myself, fall into is being overly enthusiastic about the full range of possibilities that you would be able to unlock if only you had Posit. So dashboards, multiple IDEs, easy API deployment, scheduled reports, just the possibilities are endless. Bring that enthusiasm here to PositConf, but save it for the boardroom. Initially, you know, instead focus on one unique capability. And that has to have a clear impact on your organization's bottom line.
Equally, if not more important, build your team. As open source enthusiasts, we often know how to harness the power of code-based tools, but then struggle to communicate and gain buy-in from the right people. So this is why I believe that cultivating a diversity of relationships is the key to procurement success. So be Kermit the Frog, have a creative vision and no ego, recognize the unique talents of those around you, and attract your organization's weirdos.
But in all seriousness, it's important not to over-anchor on any one stakeholder perspective. So for example, your business person is going to know how to best navigate your organizational hierarchy, gain buy-in, obtain resources, whereas your more engineering-minded folks are going to have a better understanding of where Posit and your specific data products are going to fit into the overall software development lifecycle. Not to mention, those people are probably the people you're going to need to tap when you actually need to get Posit up and running on your company's data infrastructure.
You'll also want to develop strong relationships with your customers themselves, so the people that you are building this product with and for. So I highly recommend doing extensive user interviews with these people, which is a great way not only to just clarify the problem you're trying to solve, but also simultaneously build relationships and trust with your most crucial stakeholders. And if you're lucky, you'll find someone to fill every one of these roles listed here, but let's be honest, it might just be you or one or two additional people that are willing to go to bat with you. In this case, it's still helpful to be aware of these roles so that you can know which hat to wear at any given time.
This next point is related to that core capability point at the start. So when making the case for Posit at your organization, it helps to think not in terms of what question can I answer, but what service can I provide? So i.e. create a data product. So while dashboards answer questions, data products perform valuable services, and that's ultimately what's going to get you through all of your procurement effort hurdles, which you will inevitably face.
So while dashboards answer questions, data products perform valuable services, and that's ultimately what's going to get you through all of your procurement effort hurdles, which you will inevitably face.
So relatedly, data products must be built with maintainability and robusticity in mind, which means that you need to think like a software engineer in order to win the war for limited software engineering budget. So all too often, I see other data scientists building scrappy prototypes and analyses with poor code practices, which is fine for an initial ideation, but if we ever intend to be viewed as a real component of the product development lifecycle, we need to at least demonstrate a good faith effort at applying standard software engineering best practices.
So things like a full continuous integration and deployment pipeline might be overkill, but just simply backing up your code in Git and at least thinking about things like dependency management should be on your radar.
And I have a little... My one little rant of the day is some data scientists I've met seem to think this is overly burdensome or encroaching on their creative process. And if you feel that way, I urge you to check out a talk that I link at the bottom there, which is Science as Amateur Software Development by Richard McElrath. I think he makes a great case that it's a testament to the power of science that we're able to discern any meaningful patterns at all, given scientists' chaotic development practices. So science manages to provide reliable knowledge in spite of, not because of these practices. So rant over, moving on.
Measuring impact
So my final piece of advice for anyone looking to make the case for Posit is you need to measure impact. So for us, this started with a baseline ROI analysis, which estimated the time cost savings for our organization to be about $1.2 million a year in time cost savings. And that was enough to make the case for BetterUp to procure Posit products.
But unfortunately, your journey is not over once the procurement process ends. So once installed and deployed, you'll need to continue to demonstrate the impact these tools are having on your organization. So tracking usage and collecting user feedback can be a big part of that. For user feedback, that will not only help you prioritize new feature developments, but it can also just bring joy to you on a rainy day.
So here are some screenshots of my colleagues' reactions when we first released BetterUp Insights to the People Insights team at BetterUp. So as you can see, there's a lot of joy to be had in making your colleagues' daily workflows more easy and efficient. So perhaps my favorite little nugget here was at the end of a feature enhancement request where someone signed off, thank you in advance, Posit obsessed. And I don't know about you, but I've never had a non-technical stakeholder call themselves Looker or Power BI obsessed. So I'm throwing a little shade there.
But finally, I'd like to thank several people at BetterUp that I absolutely could not have done this without. I could go on and on about them, but I'm seeing I'm at time here. And finally, Let's Connect. So you can find me on LinkedIn or Mastodon.
And if you're wondering why Rihanna is on this slide, it's because if you've seen me bopping around the conference and wondered to yourself, is she pregnant or did she eat just a massive Chipotle burrito? The answer is the former. And for me, giving a talk at PositConf is basically my nerdy equivalent of performing at the Super Bowl. So hence the meme. So that's it for me. Thank you.
So how do you best define the necessary capability when you're trying to decide to develop? That has to be developed in discussion with your stakeholders. So you would have to know the specific problems at your organization. What are the things that are people most complaining about is what I would say.