Communicating insights for impact | Noah Branham | Data Science Hangout
To join future data science hangouts, add it to your calendar here: https://pos.it/dsh - All are welcome! We'd love to see you! We were recently joined by Noah Branham, senior manager of analytics at DraftKings, to chat about marketing measurement, data science career paths, and communicating analysis to stakeholders. In this Hangout, one topic we explore is Noah's three-trimester framework for career development, focusing on how he progressed from developing basic data analytics skills to mastering predictive modeling and optimization problems, and finally transitioning to a management role. He emphasizes the importance of understanding business problems before applying data techniques and how the skills needed for success as an analyst differ from those needed for a manager. Resources mentioned in the video and zoom chat: Speak at Posit Conf 2025 → https://posit.co/blog/speak-at-posit-conf-2025/ Cascadia R Conf → https://cascadiarconf.com/ Unified Branding with brand.yml → https://posit.co/blog/unified-branding-across-posit-tools-with-brand-yml/ The Motivated Speaker Book → https://www.articulationinc.com/the-motivated-speaker-book/ Model cards for transparent reporting → https://vetiver.posit.co/learn-more/model-card.html Upcoming End-to-End Workflow Demo → https://pos.it/team-demo If you didn’t join live, one great discussion you missed from the zoom chat was all about running, including marathon advice, favorite races, and being a proud member of the Slow AF Run Club It didn't have much to do with data, but it was definitely about being a community. You should really join us live to get in on the fun! ► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu Follow Us Here: Website: https://www.posit.co Hangout: https://pos.it/dsh LinkedIn: https://www.linkedin.com/company/posit-software Bluesky: https://bsky.app/profile/posit.co Thanks for hanging out with us!
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Welcome back to the Data Science Hangout, everybody. If this is your very first time joining, welcome. I'm Rachel, I lead Customer Marketing at Posit. And if Posit is new to you, we build enterprise solutions and open source tools for people who do data science with R and Python. And I'm still going to add this in here that we are the company formally called RStudio.
I've been a host of the Hangout for a few years now, and I am now joined by my co-host here, Libby.
Hello, everybody. I'm Libby. I'm a Community Manager with Posit, helping to foster our Hangout community here. And I'm also a Posit Academy Mentor. So I help people learn R and Python to do more with data in their everyday job.
Libby's in Texas. I forgot to add, I'm in Boston. So if anybody is ever wanting to grab coffee or in the area, let me know. Our office is in the Seaport area. But the Hangout is our open space to hear what's going on in the world of data across all different industries, chat about data science leadership, and connect with other people who are facing things similar as you. And we get together here every Thursday at the same time, same place. So if you're watching this as a recording on YouTube, and you want to join us live in the future, there will be details below to add it to your own calendar.
Thank you so much to everybody who has helped make this the friendly and welcoming space that it is today. If you do ever have feedback about your experience that you want to share with me anonymously, good or bad, or any suggestions, Zoom will actually automatically launch a survey right when you end the Hangout as well. You can always reach out to me directly on LinkedIn too.
Yeah, same. And at the Hangout, we love hearing from you. It doesn't matter your years of experience, what your title is, what industry you're in, what language you work in or don't work in. And we really encourage you to connect in the chat. This is a space for all of us to hang out together. So briefly introduce yourself, add a LinkedIn profile link, maybe add your website, just some way to get a hold of you and show who you are.
Let's see, we are also going to try to grab questions from the chat. So this is the way that the Hangout works. It is an open discussion from the community moderated by Rachel and I. So it doesn't happen unless you ask questions. It's powered by you. There are three ways to do that. You can raise your hand in Zoom, we will call on you. You can jump in and ask your question. You can put your question in the Zoom chat. And if you can't use your mic right now, that's fine. You can just put a little asterisk next to it or in parentheses, like I can't use my mic right now, we'll ask it for you.
Perfect. I'm going to do one quick shout out here that the call for talks for PositComp 2025 just went out on Friday. So if anybody is interested or thinking about giving a talk, go check that out. But with all that, thank you all so much for being here and joining us today. We're so excited to be joined by our other co-host for the day, Noah Branham, Senior Manager of Analytics at DraftKings.
And Noah, I'd love to have you introduce yourself a bit here and share a little bit about your role today, but also something you like to do outside of work too. Yeah, totally. Thank you guys so much for having me. I'm a big RStudio fan, I should mention that. Yeah, so as Rachel said, I'm a Senior Manager at DraftKings. I've been there for about two and a half years. I oversee measurements of a number of different initiatives, but mostly our reach marketing initiatives. So TV, audio, and influencer, and then our monetization measurement efforts. So basically measuring how our external marketing affects our existing customer base and their types of play.
So it's a lot of fun. We have a great time, definitely a very sports-focused culture. And previously I was in the insurance industry for a little over six years. I did some work on staffing optimization, which was a lot of fun. And then for a period of time, I worked in pricing predictive modeling. So yeah, it's been, I've had a lot of fun, a lot of really interesting roles that's given me some pretty interesting experience.
And then in terms of things I do for fun, lately, I would say I've been really into running. I ran my first half marathon this last year, and then I really like to barbecue, actually. I just got a smoker in the last three years. So that's kind of a fun weekend activity for, it pairs well with football.
Background on DraftKings
Yeah, totally. I'll give a brief history on the industry. So sports betting really didn't become legal, I think, until 2018. So before that, FanDuel and DraftKings were the major players in the industry, but they had focused on a different type of game called daily fantasy sports. And so that was legal since, I think, 2012, where essentially you could set a lineup of players, and you enter a contest, and whoever's lineup has the highest points, but then you win that contest. And so that was deemed legal. But then in 2018, actual sports betting became legal, where you can bet on individual game outcomes and what we call player props, which are player-specific bets within a game. And I believe New Jersey was the first state, and a number of states, I think it's 26 total now, have been rolled out over the last now seven years.
Getting into data
That's a great question. So in college, my kind of like focus had really been to try to go into politics. And so I had actually worked for a U.S. representative going into my senior year of college, and then worked for her briefly for a short period of time right after college. And it was a really good experience, but I realized pretty quickly for me that it wasn't for me. The ironic thing was during that time, I was in charge of dealing with a lot of different data and organizing data. And that's when I really started to get into learning Visual Basic for applications. And I was like, okay, I like this aspect of the job. I just didn't like the other components of the job. And that's what led me down to data analytics.
And this was in 2009 and 2010. So analytics and data science weren't as prevalent as they were now. But coming out of the Great Recession, companies were just trying to be more data-driven and more efficient going forward. And I think that really helped spur on the data science movement. Obviously, a lot of data was being created just because of technology as well. But that opened up a lot of opportunities.
Marketing measurement at DraftKings
Yeah. So I can give an example. I will say I was told I can't go into too many specifics just for competitive reasons. But I will say essentially, companies like DraftKings or my previous employer, Liberty Mutual Insurance, spend a lot of money on marketing. And so not only is it a lot of money, but it's across different channels. And so a problem that a lot of firms, regardless of industry, are trying to figure out is what is the right amount of spend that they should be spending by channel? And so you can measure that through what's called multi-touch attribution, which is measuring direct effects of each individual marketing channel. So let's say you have an influencer on Twitter that is posting about your company. The attribution could be directly measuring the people clicking on the link and then coming to your site. Or it could be something like estimating the overall impact by understanding the synergies between the different marketing channels. And that is often done using regression and what's called a media mix model. And so that's much more high level. And so typically, firms will institute both types of methodologies to get a better understanding of how well their overall marketing is performing.
Tools and languages
Yeah, it's a great question. I would say in general, a lot of the teams use Python. My team specifically definitely has a good mix. I would say right now I have an analyst working on Bayesian models. And so we leverage PYMC for that. But a lot of our mixed effects models will use R for.
Small vs. large companies
Yeah, that's a great question. I think that was actually a very big career decision for myself. I think when I started out, I really wanted to work at a smaller company and get a lot of experience in a lot of different areas. But I found out that I was, being a jack of all trades is being a master of none. And I think for me, that was good for a short period of time. But I ultimately started to get asked these very difficult questions by board members of these different startups. And I couldn't answer them because I just didn't have the skills at the time. And that's what really actually pushed me to go from a more traditional analytics role to go into data science at a larger company. Because not only were there resources, and by resources, I mean other really, really smart people who can help me out, who had PhDs and help train me, but resources in terms of being able to fund going to grad school and pay for courses and whatnot. And that really helped elevate my career. On paper, it might have looked like a slight step back, but it was really just a way of investing in myself.
Yeah. I think at first, I really tried to learn as much as I could on my own. I leveraged data camp a lot. I talked to my friends who were a lot more technical than myself. But I realized you can only go so far doing those things on your own. And so that was one of the benefits of going to a place like Liberty Mutual, where there was a lot of really smart people. We had a data science community, even within Liberty Mutual, that we had resources that we could talk through. And then one thing that I did that I actually really tried to focus on was I looked at a graduate school program's entire agenda, and I tried to learn as much as I could. So then when I went to graduate school, it wasn't as much of a chore. And some people in the graduate program struggled. But for me, it was more just filling in a lot of those gaps.
Three-trimester career framework
Yeah, totally. So I would say the first trimester was really just getting familiar with working in the corporate world. And so for me, a lot of that was how to conduct and lead meetings, how to get your basic data analytics skills in order. So understanding, at the time, the tool of choice was really more Excel for me, just based on where I worked. And then getting into SQL, learning how to create presentations, and actually share those presentations out and create clean decks. That was really the first trimester for me.
And then that's where, like I mentioned before, I was starting to run into those struggles with how to answer really difficult business questions. So the second trimester was really getting into the data science world, understanding predictive modeling and optimization problems, and how to solve them. And I think a big thing is understanding the business problem, and then knowing what type of technique you need to do to then tackle that problem. I think a lot of people really struggle with that. And I think that's a really powerful thing to be able to understand. So I spent a lot of time doing that. And I was almost pushed into management roles eventually. And so that's really my third trimester, which is the manager trimester, which I think was the biggest adjustment. I think what gets you success as an analyst doesn't necessarily get you success as a manager. So you have to reframe your goals and time management, and how you spend your hours during the day.
Communicating analysis to stakeholders
Yeah, that's an excellent question. I think your problem is actually a very common problem that I call it when the rubber meets the road in data science. I've seen really excellent data scientists build fantastic models, but they haven't really been able to have the impact that they need to be successful in the role because they can't communicate their work. Somebody I think recently said to me, your analysis is 50% of it, but communicating your analysis is the other 50% as well. So that's an excellent question. I think there isn't an exact resource that I have for you, but I can tell you what I do, which is I try to find someone on our team who is not familiar with our work, and I ask to spend like 30 minutes with them to just share our presentation with them just to get their feedback because it will all be new to them as it would be to an executive in my situation. So that's one tool I've leveraged going forward that's really helped.
Somebody I think recently said to me, your analysis is 50% of it, but communicating your analysis is the other 50% as well.
I think practice in general is just really important in terms of getting your information out. And then I think getting as much feedback as possible from your managers is really important as well.
Big wins and impact
Yeah, totally. I'll actually give one from a previous role. When I was at Liberty, staffing was like a pretty large expense for us specifically on the operations side. So just to give you kind of like an idea of how it goes is every insurance company has adjusters who essentially handle claims. And the types of claims require different types of adjusters. And as you probably are aware of like the Los Angeles fires, for example, like those require certain types of adjusters to handle those claims. And there would be like a leaky roof from a storm in New England. And so we hire thousands of or that Liberty hires thousands of adjusters. So when there's too many claims in a given area, they have to hire essentially what are called temp adjusters. And there's like a cost both from like a customer service perspective and an actual like financial perspective. So those temp adjusters are not, they don't do as good a job and then they have much worse customer satisfaction ratings.
So essentially in year one, what we did was we kind of took over the model, I should say in quotes, that they use to staff in these different locations. And then we essentially built out an optimization to choose the optimal number of adjusters based on like a pretty fixed amount of what we call demand, which was claim volume. So that was like our big win for year one. And then our big win in year two was we were able to basically change the demand forecast by MSA. And so basically we were able to create a bunch of different scenarios for like different weather events and types of weather years, like El Nino and La Nina. And that allowed us to create a wide range of actual adjusters that we needed in these different MSAs. And that ultimately resulted in about $100 million of savings per year. So I think that's my biggest like impact win probably that I can think of recently.
Skills Care Squad and internal learning
So Skills Care Squad was actually started by another director, and it kind of is very much like an analytics community within DraftKings. So what it is, is we essentially have a group of people together that I lead, or I've led since last May. And we essentially create like a curriculum for analysts to learn. I should say managers as well. It's not just for analysts, but we focus on specific topics that we think the analytics org would benefit from. And we basically create a curriculum. So we're actually just laying that out for the next year. I think in the last year, we focused a lot on technical skills, like k-means clustering, regression, data visualization. This next year, we're going to focus a lot more on presentation best practices, building out better presentations. We have a series called Communicating for Action, where we take different versions of a previous presentation, and we actually share each iteration. So there was the iteration of the presentation that was for our direct stakeholders, like manager, and in my case, manager and marketer. But then that needs to get escalated to VP or executive level. We have a different version of that presentation, and we just talk through the changes that we made and how the results would change.
Yeah, so I didn't actually start it. A previous leader had started it. And we had been trying to rotate people off of the squad just to give people other opportunities. There was a lot of interest there. And so I was asked to take it over in May. And we actually doubled the squad size for 2025, just because there was a lot of interest. And we wanted to basically get more trainings out. But I think I was able to choose the team last year, just as I took it over. And it was a lot of people that I had trusted and I knew were going to be able to deliver on a lot of the technical presentations. And this year, I specifically wanted to rotate folks out, one, to get to know different people. But also, I wanted to balance it out with some folks who had skills in areas that we hadn't covered as much.
Yeah, it's a great question. I would say that varies at the probably director and manager level. So for my team specifically, I actually have specific names for them. But it's essentially an office hours. But I do it on Friday afternoons, and I always encourage people to bring a sweet treat. It's kind of like an informal thing. So I'll bring a cafe mocha and we'll review a model that somebody's worked on, and somebody will share a model. So I like to do informal brainstorm and knowledge share. I also kind of informally became this modeling person that people on the team can go to. So a lot of times, people will just reach out to me, and we'll set up time to at least brainstorm how I would approach the project. I think your question's a really good one. I think instilling that openness among leaders. When I say leaders, it doesn't have to be a manager. It can be a lead data scientist or a lead analyst. And setting that up so that you have infrastructure so that people have the time sets your team up for success in the long run.
Data infrastructure and tooling
No, we don't use C++ or Rust. Our team strictly uses Python and R and obviously SQL. I don't really consider that a programming language. That's a database language, but that's basically the infrastructure that we have. The way I approach things though is, and it kind of answers both questions in a way, is we can use whatever tools do the job. So it's not necessary to use Python for very quick analyses sometimes. So I'll use Excel for that at certain times. But then most of the time, because my team does a lot of modeling work, we will spend most of our time in R and Python.
Yeah. I mean, we use Snowflake for pretty much everything. And we are bringing on Databricks as well for a lot of our work.
So Snowflake, we primarily use as our SQL infrastructure. So if we need to pull data or write a query, that would be how we would use Snowflake for. Databricks, you can also use Snowflake, but you can also use R and Python. And because it's cloud computing, you can leverage parallel processing to deal with very complex problems and run the code in a much more efficient way in terms of runtime. There's a cost every time to use Databricks though. So we tend to be more conservative there just because if you're using a lot of cores, it's going to cost the company a lot of money.
Ensuring analysis has impact
Yeah, definitely. I think that's an area, to be honest, I've really focused on the last three years of my career, three or four years of my career, and I think it's really important. So I've seen, like I mentioned earlier, I've seen a lot of really strong technical analysts build out fantastic models, but the actual results didn't have impact because they weren't able to communicate it effectively. And that could be a variety of different reasons. That could be because their deck was sloppy and leadership saw it and looked at it for two seconds and was like, okay, I'm just not going to trust this. Or it could be because of a slight mistake in the presentation. Some leaders will immediately write off the presentation.
So I think the way to first deal with that is have very strict regulations around what goes into your presentation. So formatting, like little things like having all the fonts in the same size, having all the boxes on your slides be in the same format, and really being almost OCD about that. They were actually really strict about that at Liberty Mutual when I worked there. I accidentally, I remember, used the slight one shade off in yellow and everybody called it off on one of the bar charts. So I think really sticking to those makes it look just more professional and creates belief. And so I think I undervalued that earlier in my career, so that really helps. I think the second thing too is then having a sound story around your analysis. You can't just be sharing model results. It actually has to be translating that into business action. Like one of the best courses I took in grad school was actually called Strategy and Analytics. And it was basically just 10 case studies of the teacher giving us, the professor, excuse me, giving us regression results in a situation and then having us focus on turning that into a presentation and sharing it out and giving our recommendation. So it was less about the actual process of data science, but more just like actually turning it into impact.
Career advice
I think there's a... Fortunately, I have really good mentors and leaders throughout my career, so I've received some very good pieces of advice. But I think the one that really stood out to me was if you are the expert in the room, and I mean on average, it's probably time for you to move on from your current role. Because there isn't a lot of people there to really help you grow and learn. I've been the opposite of the expert on certain teams, and those are situations where I might have gone home and I had imposter syndrome every single day, but I did learn the most during that time period. And so I think it was an important experience for me to have.
If you are the expert in the room, and I mean on average, it's probably time for you to move on from your current role. Because there isn't a lot of people there to really help you grow and learn.
But really all it was, it was data sets from a business, and he made a regression model, and he put it in a PDF and just gave it to us and said, hey, translate what this means for the business. You could do that. You could probably go on Kaggle, get data sets, build a model. It doesn't necessarily even have to be right for this situation. You can just assume it's correct. Personally, I would try to make the best model possible and then try and say, hey, what does this mean? And I think it's important to take a step back from the actual modeling process when you do that. I think a lot of analysts just get bogged down in that and forget about the business impact and the business implications of their work.
I think it really helps to actually go back to your exploratory data analysis to help tell the story. I think a lot of people are trying to lean on the model results, but if you go back to some of your EDA, that can actually help provide a visual representation. And a lot of people are visual learners, so that can really help tell the story.
Communicating uncertainty
I think it's a great question. So, I think it's one in my situation where it was actually not, yeah, somebody actually just said it, all models are wrong, but some are useful. That's a famous quote from George Box, and I think I actually try to reference that in a lot of my presentations that models are essentially just guessing using statistics, and a lot of the situations that we're in, there'd be no alternative way of measuring it. So, you have to first have buy-in from your leaders who are not in data science that modeling is the solution, and I think one mistake I guess I would say I made earlier on was when we presented model results, we presented basically the average results, like on average, the model says X, Y, Z. We've gotten more now into range of outcomes, and I think the range of outcomes has helped build more confidence in saying like, hey, the model outputs a range between this value and this value, and on average, we think it's this, and I think that's helped get buy-in from a lot of leaders rather than point estimates.
Great, thanks. I also think somebody else said something in the comments that I thought was very good too, is starting with the so what, and then it's almost kind of like reverse engineering it. You start with, hey, this is what our model says, and then you can get into some of the details from there. I do think you can lose higher level leaders if you start to get into the minutiae of your model, but I keep all that information in the appendix, and I oftentimes reference it as the presentation goes on. I had one instance where I was presenting to our CMO, and we got into a relatively complicated concept, but I had one box plot that was like there to save the day to help explain the difference between two groups that we were looking at, and so that really helped in that situation.
Becoming a manager
I think the most surprising thing was just the volume of meetings and how your day-to-day changes. I went from basically in a world where I would have maybe one or two meetings a day, and I could just sit and build models, to a world where I had meetings from 9 a.m. to 5 p.m., and I had a bunch of people messaging me all day needing stuff or asking questions. I think that change was just very different. My ability to spend time on modeling went down significantly. When I work on models now, it's typically before work or after work or just in my personal time because I'm interested.
Then I think from there, that was a big surprise. I think that forced me into time management, forced me into really thinking about what it is that I want to be doing on a day-to-day basis, which has changed my responsibilities at work a little bit more to just focus on modeling now, as opposed to some of the more traditional analytics that requires more day-to-day work with our stakeholders.
Yeah, absolutely. I think I was pretty nervous, to be honest, to be a manager for the first time. I read a bunch of management books. I think a lot of the data management books are really interesting, but a lot of those management best practices were created before analytics and data science were a thing, so you have to apply your own spin to it. I think first and foremost, you have to have management principles. You should really, really think about those principles and how you can apply those principles. My first job as a manager, I had very junior employees, and so my principles were very different in that role. My number one principle at that time was set my team up for success. If I ever had a situation where I was at a crossroads, I would think back to that principle. How do I set my team up for success? What action do I take to set up my team for success? Some of the time, that would be like, okay, maybe I shouldn't get involved as much and let this analyst who's trying to become a senior analyst take that work on his or her own to prove that they can do it on their own. Earlier on in that tenure, I might have said, hey, I have to step in because this is a junior employee. I think deciding on those principles is really important, and you're going to have different principles for different situations. The role that I have now, I have all senior employees under me, so the principles are not about setting up for success. These principles now are much different.
It's different in this situation because of business need, but it's also different because while they all might be at the same level, their technical expertise might be different, and what they want to do in the long term might be different. I have some employees who've been only at DraftKings their entire life, or their entire career, I should say. For them, they're going to benefit most from my non-DraftKings experience and learnings from that. Then I have some who the work for them is just very easy, and so I just need to keep them interested by giving them more and more challenging projects. It really just varies person-by-person in this situation.
Keeping a career journal
A lot of it, honestly, is what not to do. I'm a little bit of a cynical person, admittedly, so what I've noticed is when somebody makes a mistake, it sticks out to me, so I try and keep a note of like, hey, this didn't go well. Other people in the company were upset about this. There was a lot of internal talking about this specific thing, so I'll take note about, hey, this is what not to do. A lot of that is also in terms of management. If I had issues with the previous manager or I didn't like how something was communicated or something like that, I'll take note of how that communication happened just so that I don't do that in the future. Then I think back to what I have enjoyed in my roles over time, and I think about this periodically. A lot of times, this will be when I go get back from a vacation and I'm fresh and I'm just thinking, hey, what makes me happy at work? I try to just take a note of that. I don't do that every single day, I would say, but if something really stands out, I'll add it to my list.
A lot of times, this will be when I go get back from a vacation and I'm fresh and I'm just thinking, hey, what makes me happy at work? I try to just take a note of that.
Thank you so much, Noah, for taking the time to join us today. Thank you all for the great questions as well. It's so nice to see you all. Yeah, thank you guys so much. Really appreciate it. Take care.
