Integrating video & data in sports analytics | Arielle Dror | Data Science Hangout

Transcript#

This transcript was generated automatically and may contain errors.

Hey there, welcome to the Paws at Data Science Hangout. I'm Libby Herron, and this is a recording of our weekly community call that happens every Thursday at 12pm US Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience.

Can't wait to see you there. I'm super excited to welcome our featured leader today, Arielle Dror, Director of Data and Analytics at Bay FC, which is a soccer team in the US National Women's Soccer League. Arielle, welcome. Thank you so much for being here.

Hi, Libby. Thanks for having me. I'm really looking forward to today.

Me too. Okay, I would love it if you could give a brief background about yourself. Tell us a little bit about what you do, and also something that you like to do for fun.

So I am currently the Director of Data and Analytics at Bay FC. We're one of the newer expansion teams in the NWSL, National Women's Soccer League. I work on the football ops side, where I work to integrate data into all of our decision-making processes, whether that be recruitment, tactical analysis, game prep, match review, salary cap analysis. If there's something that the sporting side is working on, I'm trying to integrate data to try and make our decision-making processes more informed.

And what I like to do in free time, well, I have a puppy. His name is Bingo. He takes up all my time these days. He's not here right now, unfortunately, otherwise I would have him make an appearance, but maybe I'll post him on Blue Sky after. I'm also an avid knitter, currently working on my second sweater of the season, and I also really enjoy swimming.

Quantifying intangible player traits

So we do have an anonymous one that says, what traits in players do you wish you were able to quantify, and do you think we'll ever be able to quantify traits like drive in athletes?

So I will say, the data that we have in soccer is slowly evolving, and it might be helpful to just give an overview of what we have available to us right now. So most of the data we have is event level data. So if you think of a soccer game, there's one player that is usually on the ball, and then there are every other player on the pitch is not on the ball. And so this event data only collects on-ball information, which means we are missing like 95% of what happens in a game. So we have things like passes, shots, defensive actions, things like that. We don't necessarily have off-ball runs, where people are in space, et cetera. And so most of what you can do in soccer is actually quite limited.

We're now getting to the point where we have that off-ball information, where you can actually quantify off-ball impact. So we're slowly getting to the point where we can quantify a lot of things. I do think that often in recruitment decisions, we start going into these personality level questions, and things like drive, or dog, or how fancy they are. I would love to be able to quantify those things to be able to actually give an informed discussion to actually drive what we're doing. So I might actually just take the example that you gave here, which is drive, or just tenacity. That's something that often is really talked about in our discussions that I really can't contribute to, and I have to try.

It's tough. I think that anybody who works with people data, data about people, qualitatively or quantitatively, will say that anything that's a personality trait is extremely hard, and the results are dubious. Humans are complex systems, and anything that humans participate in are complex systems. I don't think we can ask, based on how recruitment works, we can't give them psychological evaluations either.

Integrating data into tactical decisions

Yeah, that's a great question. So I'm really lucky at Bay. We have two video analysts that work under me who do a lot of the work in terms of integrating data into our tactical decisions and sort of in our weekly scouts. And one of them is particularly data driven. He worked with the U.S. Women's National Team for several years, won a World Cup with them. And so they had a lot of data folks on their team. So he's actually pretty data literate. And so he, whenever I build our weekly pre and post match reports, he will take a lot of that data and integrate it into our video scouts that go to the players and to the coaches. And he uses it to inform some of the tactical recommendations that he makes each week.

We will occasionally put like data in front of the players to sort of emphasize a point. We went through a period this year where we were dropping points where we shouldn't, like we're probably, the probability of us dropping the points we did was like less than 1%. So we put it in front of them to sort of emphasize the point that they were doing well. So we do it a lot. We do a lot of work like using data to increase the confidence of our players.

But we do use it, like I said, for tactical decision-making too. So a good example is we have a pre-match report that I have automated that goes out every week. And one of the plots that I show is like who is the top chance creators on the team based off of a lot of different things that we do. There were two players who nearly all of their chances were through these two players who were not forwards. And so we designed an entire tactical plan around keeping them off the ball. And they did not get on the ball. And then we were able to win the game. And we were able to also use data afterwards to check our process. So we were able to see that they actually did not have any chance involvements. So that was like probably the cleanest example we have of where data is used to inform the decision, to find the information, inform the decision, and then actually evaluate after whether it worked.

So that was like probably the cleanest example we have of where data is used to inform the decision, to find the information, inform the decision, and then actually evaluate after whether it worked.

I do endorse working out loud. Even if it doesn't turn into a job, you make lots of really great friends, which is maybe even better.

Recruiting challenges and the passion penalty

Honestly, I think the pay is just not on the level of what we would expect for people with technical skills, and that's something we're trying to change our pay. Also, we often want people who have some level of sports experience. It's really hard to get that experience, so working out loud is really helpful. Those are probably the two big ones. I find the most talented people that I would really want to come work here are just not willing to work in sports, which is super fair. It's a really hard, it's definitely not your typical tech job or typical data job.

I would say definitely. Actually, let me backtrack. I majored in data science. I also majored in political science. I was really interested in doing data science for social good before this. I feel like every job I've had has paid a passion penalty to some extent. I would say it's true. It's definitely true. I think it's getting a lot better compared to where it was when I started. It's definitely gotten better, but there definitely is some level of passion penalty, especially because those jobs that you're comparing it to, the work-life balance is just not the same. You're usually working in a smaller team.

Something that I really love is that I get to see my work actually put into play every single day. I know I'm making an impact with the work I'm doing. There's also the fun stuff of working in a team. You don't have to be experienced all the time, but I travel with the team. I've gone to see the world, quite literally the world, and meet really interesting people, work on really novel ideas. Those things I don't think I necessarily would have had the opportunity to do in a different industry.

Technology and modeling in sports analytics

I would love to be doing really fun machine learning and cool novel work like that. Unfortunately, we have a lot more low-hanging fruit that we need to meet first before we can do really cool stuff. I actually don't do a ton of modeling in my day-to-day. It's a lot of data engineering and a lot of much more simple data analysis, data viz. One day, I would love to do something like that. I do also think we probably need to get a larger quantity of data or a video before we could feel comfortable doing something like that, but it sounds really cool. If someone does that publicly, tag me because I would love to see it.

Data availability and live data

In terms of live data, we get a little bit from the league from Opta, mostly just in the form of a dashboard where there's nothing we can really ingest, though that might be changing next year, fingers crossed. All the data we get is post-game. So we get stuff from our data providers ingested via API. We get more data than what is available publicly, but it's not from the league. So some teams in the league do not work with those data providers at all.

Home field advantage and sporting myths

So we don't do a ton around audience home field advantage, etc. Most of the stuff we use there is just stuff that's publicly available because I try and use as much as publicly available as possible because it's just me. There's been some stuff around different tactical decisions that have happened or in-game moments. I want to share them, but I feel like they're kind of competitive advantages, so I don't want to share them too much that have been interesting to prove me wrong. But I will say a lot of those types of questions, we use publicly available information. Just again, there's not a lot of time to do some of those analyses.

I know that this sounds like a fun thing for like fans to tackle. There are some questions that like if you are curious and if you think this is a fun thing and you can find data that is publicly available, it would be really fun. I'm sure there are some home team advantage analyses out there for some sports, right? I think that would be super fun.

Breaking into sports analytics

So, first of all, I'll say that a lot of clubs now also do have BI roles. So, that is totally an avenue that you can take. I do think like in terms of networking, working out loud, et cetera, there's a couple conferences that I've gone to that I've found particularly friendly. America's Soccer Insights Conference Summit. I can't remember the full name. In Houston, I went last year. It's only the second year. It was the most friendly conference, except for maybe Posit Conf that I've ever been to. And they especially, they like particularly plan things so that industry professionals are talking to people that are interested in breaking in through like lots of breakout sessions.

I've gone to a lot of like women's sports data type conferences. I've also found those to be really friendly. So, women in sports data, it didn't happen this year, but in the past, it's been super friendly and lovely. So, I highly recommend following people on Blue Sky. Like following people on Blue Sky is often how I did most of my networking and just talking to people. In terms of like projects that people do, I find like the stuff that is quite interactive and not just like a single report or article. It's the stuff that draws me to profiles. So, packages, dashboards, that sort of work is, to me, just much more compelling than like a single analysis.

Wearables and athlete performance data

Yeah, that's a great question. So, to answer your first part, we have a sports scientist on staff who handles all of our sports performance data. We're slowly getting to the point where we can collaborate together more, but she handles most of that day-to-day. All of our players wear GPS units in-game and in training for us to track things around sprints, accelerations, decelerations, heart rate, all that kind of stuff. And in some leagues, that's required. In other leagues, it's voluntary. It's all up to the CBA. I think in our league, it's required. I've never seen any player not wear one. So, that's how they collect most of their data. The players also do fill out daily wellness surveys around sleep and whatnot. I don't think we're taking any data from their personal wearables. I think that might be a privacy violation.

Motion capture and tracking technology

Yeah, so I will say I'm not familiar with any 3D motion capture through sensors at all, so I don't want to talk about something I don't know much about. It might exist. There's tons of 2D motion capture stuff. So there's obviously the GPS units we've talked about, and then there's also optical tracking systems where the cameras are set throughout the stadium, and they're collecting all the information from all the players and also the ball. And then what we use more often is broadcast tracking, which does the same thing but uses broadcast video. So it's basically using computer vision to track the players in the ball that are on camera. The challenge there is that in some leagues, the broadcast quality is not as good, and they have to use models to impute the off-ball players, which is a challenge for many reasons.

Handling false positives and data-video collaboration

Yeah, I mean there's plenty of moments where that can happen. So we always try and, like every half six months, we look at our reporting and see where we were right, where we were wrong, what we might need to add to capture things. Sometimes things happen in games that are not quantifiable in data, or at least not easily. And that's why, you know, going into this, I only wanted to use data to make all of our decisions, because that's sort of how I operated in my last job. And I still live in that camp a lot of time, but I've also learned that using video analysts and using their expertise in collaboration with the data is really like where our best work is done. So it's a lot of interrogation of where those things might diverge, fixing things, seeing where things might be breaking, and also accepting that people, like everyone's best skills work together to give us that answer that we're looking for.

Building the analytics stack from scratch

Someone asked how I picked the analytics stack, and I really like that question because I was in a pretty interesting situation when I joined Bay, where I joined in October, and our expansion draft where we picked players from existing teams was in December, and when I joined, we had absolutely nothing. We had no contracts signed except for one single data contract, and we were already making decisions on player transactions, which is really stressful. So I was tasked with deciding really early what that stack was going to look like, and so I did my best to try and find things that I knew were really flexible because I didn't really know how we were going to grow. So that's why we ended up with Snowflake, for example. I felt of all the warehouse solutions, that was the most flexible for what we wanted at the time.

At first, I just built R pipelines, which is still great. I've worked in full R stacks for a long time, but I built everything working off of my computer. There was no orchestration. Nothing was deployed, really, and so in the second year, I was finally able to build a more coherent, error-proof tech stack using Astronomer and Python and R and a lot of different things, but the point I wanted to make here is that I worked really hard to build a flexible tech stack because I didn't know how things were going to work and also was okay with having something really crappy at first with the knowledge that I was eventually going to go back and make something more robust. It's okay that things aren't perfect right away. Sometimes things just need to work. Sometimes you're already making decisions without any data and you need things to give you context.

Players optimizing for stats and communicating analytics

That is such a fun question. First of all, I do want to say we have some players that are super interested in data. One of the cool but also sad things about women's soccer is that most of our players will have to have jobs after they retire, and so we have players that have majored in computer science or math or are just interested in data, so that's been really cool to give them fun high-level intros on what we're doing. I'm not worried about them trying to optimize for stats because we use lots of things in our decision-making processes, to be honest. I'm more concerned about them — I mean, it's good that data is publicly accessible. I would love them to just use us as the source of truth for data because the reality is that we can contextualize it more, so that's my bigger concern is that they're just trying to get data information without context from other people.

Yeah, well I think, to me, the big thing is meeting people where they're at, right? Like, I spent the whole first six months of my job, or even more, building dashboards and reports with, and you know, I would talk to them and be like, do you understand this? And they would be like, yeah, I understand, but it turns out, like, I wasn't meeting them where they were at. I was looking at things with the total data lens. And once I found out that I had to use the video to contextualize what I was doing, they opened up this, like, communication that we didn't have before. So yeah, meeting people where they're at and just working with them slowly. Eventually, hopefully, they feel confident enough to tell you when things don't make sense to them, and I make it really clear to them, like, that I am a thought partner. Like, I'm not here to be, like, the, like, nerd in the corner that's telling them what to do. Like, I want everything we do to be really collaborative and that we build together, because they also have expertise, you know? I learn a lot from them just as they learn from me.

I am a thought partner. Like, I'm not here to be, like, the, like, nerd in the corner that's telling them what to do. Like, I want everything we do to be really collaborative and that we build together, because they also have expertise, you know?