Warren Hearnes @ OptiML AI | Data Science Hangout
We were recently joined by Warren Hearnes, Founder of OptiML AI to chat about 30+ years using optimization, ML, and AI to solve significant problems for businesses (BestBuy, Home Depot, UPS, and more). Speaker bio: Dr. Warren Hearnes has held many roles in the areas of optimization, machine learning (ML), and artificial intelligence (AI) over his 30+ year career. He recently started his own company, OptiML AI, operating at the intersection of optimization, machine learning, and artificial intelligence. Prior to that, Warren was Chief Data Scientist at Best Buy, where he led a team of 45+ data scientists that elevated the use of AI, ML, and optimization across the enterprise, including marketing, labor, supply chain, and customer service. His team collaborated with technology partners to build the environment of tools, access, standards, and platforms that enabled increasing scale, sophistication, and impact of data science across the enterprise. His career includes significant roles such as Chief Analytics Officer at Cardlytics, where he played a pivotal role in the company's growth to a multi-billion-dollar entity, and positions at The Home Depot, UPS, and Lucent Technologies. ________________________ ► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu Follow Us Here: Website: https://www.posit.co LinkedIn: https://www.linkedin.com/company/posit-software To join future data science hangouts, add to your calendar here: https://pos.it/dsh We'd love to have you join us in the conversation live! Thanks for hanging out with us!
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hi everybody, welcome back to the Data Science Hangout. I'm Rachel Dempsey, I lead Customer Marketing here at Posit. And you know what, I just learned that some people are actually hearing about Posit through the Hangouts, so I wanted to add a little bit about Posit as well if it were new to you. We're the open source data science company building tools for the individual teams and enterprises. And so I'm so happy to have you joining us here today.
The Hangout is our open space to hear what's going on in the world of data across different industries, get to chat about data science leadership, and connect with others facing similar things as you. And so we get together here every Thursday at the same time, same place. So if you're watching this as a recording on YouTube in the future, and you want to join us live, there'll be details below on how you can add it to your calendar. Just a quick note to make sure it adds it for you at 12 Eastern time.
If this is anybody's first Hangout, I would love to see you say hi in the chat just so we can welcome you in here as well. We're all dedicated to keeping this a friendly and welcoming space for everyone and love hearing from you no matter your years of experience, titles, languages that you work in, or industries. You could be a part of the party happening in the chat. You could also jump in and ask questions or provide your own perspective.
So you can raise your hand on Zoom. If you're wondering how you do that, it's in the reactions bar in the Zoom bar below. And I'll call on you to jump in. You can put questions into the Zoom chat and just put a little asterisk next to it if it's something you want me to read instead. And then third, I see Curtis just shared the Slido link where you could ask questions anonymously too.
But with all that, thank you again so much for joining us here today. I'm excited to be joined by my co-host Warren Hearnes, founder of OptiML AI. And Warren's held a variety of data roles across his career from chief data scientist at Best Buy to roles at Home Depot, UPS. And it's fun that this is actually the first time I'm getting to meet you live on camera here as well. So I'd love to have you introduce yourself first and maybe share a little bit about your new company.
Warren's background and career journey
Sure. I'm glad to be here. So I'll give a plug for Posit. I've been using RStudio for many, many years. So if you don't know about Posit and RStudio and some of the other things that I'm sure I don't know about, talk to Rachel or to Jake. I also like to see, it's a great crowd. I see Curtis is on the treadmill. I like it that he's multitasking and getting some exercise in.
So I'll give you all a little bit longer background than I normally do for a discussion. But I've been in what we know as data science for, I guess, it's 2024, so 32 years now, not including some time in operations research in the Army. So I've always loved using math and data to make better decisions. And my undergrad was at West Point. I started in 1985 and graduated in 1989 and got a degree in mathematics and operations research.
And so operations research is the more traditional things like image programming, linear programming, statistics simulation, things like that. So thought I was going to spend a career in the Army. Did all the fun Army stuff like airborne ranger, field artillery, got to blow some things but decided to get out in the early 90s. I got in and got out in 1992.
And that was before we had any search engines or anything like that. I knew I wanted some type of job that used math. So I was visiting someone in Portland, Oregon, and I actually had to take a taxi down to Portland State University to look at a physical bulletin board with physical job postings up on it. And I saw that all of the job postings, and there weren't a whole lot, but all of the job postings in industry required a master's or a PhD. And so I thought, well, maybe I'll get a master's.
I applied to Georgia Tech, got accepted into their program, knew nothing about grad school, got to Atlanta from Hawaii. That's where I was stationed. So I got to Atlanta, found out that they don't fund master's students. They asked me if I wanted to be a PhD student. And I said, sure. So I just jumped into a PhD program. And this is where one of those just chance meetings changes everything.
So at the time, this was the fall of 1992, machine learning was around, but it wasn't as popular as it is these days. There were only two professors at Georgia Tech who were doing funded machine learning research, and one of them was in the industrial engineering school at Georgia Tech. And he was working in reinforcement learning for robotic control. And I was already studying neural networks, fuzzy systems, and things like that. Turns out it was a chance meeting got me into that research. And so I started doing reinforcement learning for robotic control using adaptive dynamic programming and things like that throughout the mid-90s.
Now, machine learning wasn't as popular. So a lot of the people said, well, if you can't prove it has the optimal answer, then it's not really worthy of publication and some things like that. So I decided, I also met a girl from here in Atlanta. We were getting married. I needed to buy a house, so I needed to have a job.
And so I started working at Lucent Technologies. And there at Lucent Technologies here in Atlanta, at the time, we had the world's largest fiber optic cable factory. So we were using integer programming, so mixed integer linear programming to schedule all the work in the factory and to assign the raw materials to all of the orders. I mean, you were really using a lot of very complex math to make that factory run much more efficiently.
Then spent about six years doing that. UPS is also headquartered here in Atlanta, so got a chance to move over to their corporate headquarters where I started doing machine learning once more. So we were doing forecasting. We were doing fraud detection. We were also using some integer programming to move the empty, relocate some assets around the United States. So spent eight years doing that.
Moved over to the Home Depot, which is headquartered here in Atlanta, and that was my first foray into marketing analytics. So our team was in charge of all the direct mail and all the email that went out. That was in 2011. So if you got something from us back then, that was our team. It's how do we target? Who do we target? How do we execute the marketing campaigns? How do we measure them? And how do we get a good return on investment from that?
That got me noticed by a small startup in Atlanta called Cardlytics. So I switched after a year at Home Depot. They still do have a great business model. Work with the banks. The banks know really the best predictor of your future purchases is your past purchases. A lot of companies will use demographics and things like that to help as a proxy for what you're going to buy in the future, but the best predictor really is what you bought in the past. And so with that partnership with the banks and eventually got 50% of all the card swipes in the United States and the UK, we were able to grow that company, take it public in 2018.
And I built up a analytics and data science capability at that company that really did some amazing things. And T. Roo was on here. He was leading our team in India, doing some very amazing things with NLP and trying to take these merchant strings and classify them. So great company with some reorgs and some changes. Left that company in 2022.
And got a chance to become chief data scientist for Best Buy. Best Buy was growing a technology and analytics hub here in Atlanta. So I helped start that. We were basically applying optimization, machine learning, AI to a variety of problems around the enterprise, whether it was supply chain, labor, customer segmentation, media mix optimization, call centers, store operations, you name it.
So then about, maybe about one month ago, almost to the day, we reorged at Best Buy. I got a good package. I left that company. And right now I've technically started this company called Optimal, and it's spelled Opta, M-L-A-I, OptiML AI, because it's the combination of the things that I've been doing for my entire career, optimization, machine learning, and artificial intelligence. I haven't done much with it yet because I've just been playing a little golf and doing some traveling, but that's really going to be my focus. I want to, if I'm doing something either part-time or full-time from this point out, I'm close to retirement, then it's going to be helping companies learn how to use the newer things that are data-based, like machine learning and artificial intelligence, and combine them with some of the more traditional methods, like energy programming.
AI hype cycles and historical context
Yeah, I think in some ways it's similar. So AI has had its ups and downs for many, many years. You know, I think in the 80s, that was like an AI winter, and then things will happen. Something significant will happen, like what's happened with Gen AI lately. I want to make sure that people in this field don't gloss over the things that have happened in the past. And I'm not talking about the past months, couple of months, or a few years. I'm talking about decades of thought and research.
So I like to say that, you know, in January, there was a paper called Steps Towards Artificial Intelligence. And in that paper, they talked about basically five things. And that's the problem of search, the problem of learning, pattern recognition, induction, and planning, those types of things. And so those are all some, you know, every single one of us are working in search and pattern recognition and learning. But the point that I make is, and I've got a copy of it right here, because I do this example all the time. That wasn't January of this year, that was January of 1961. That was Marvin Minsky in 1961, talking about reinforcement learning and pattern recognition, and everything that we are still working on today, over 60 years later.
That was Marvin Minsky in 1961, talking about reinforcement learning and pattern recognition, and everything that we are still working on today, over 60 years later.
It's not to say that we haven't made some fascinating leaps and bounds. But when I tell people that I was working in reinforcement learning in the 90s, a lot of them are surprised that we were doing that in the 90s. Well, we were doing it in the 50s and 60s. It's just that we're doing it at a much higher scale now.
So I think right now we are in a very, you know, we're riding the wave of gen AI to talk about what we can do with it. And then there's going to be that, you know, standard hype cycle. There's going to be, let's get it into practice. There's going to be a big cost. We're going to figure out how to do it better. There will be a plateau. Then there will be something else. And I guess my last point on that would be, I had heard about large language models from many, many people. So it's not as if just one thing came out. You know, in your career, you're going to see a hundred ideas and maybe one or two of them are big ideas. So you've got to learn to sift through those. And to be honest, there are a couple of people on my team at Best Buy, Jimi Hendrix and Will Armstrong, they were telling me about some of the new advances in large language models. And I was more like, yeah, I've heard about those advances in the past and they haven't really panned out. And it wasn't until basically GPT 3.5 that I saw a big step change.
Job requirements and generative AI skills
Yeah, thank you. Yeah, because like, as you were mentioning, like, yeah, like GNI and like all the more recent developments. I'm curious. So my background is in like bioinformatics and I've been in the field for quite a while, like got my master's 14 years ago. So I was doing everything around like data science, machine learning, computational biology for a while. And now when I'm looking for a new job, what I see is that a lot of job descriptions, they suddenly have all these requirements of like, if you don't have, I don't know, a PhD degree in NLP or like know how to do all this stuff with, not just like, of course, like not just using, but like training, fine tuning, all of this with like GNI LLMs. And I find myself at this point when I'm like, okay, should I invest? I don't know how many hours of my time and maybe like take another course. So at least that I can say that I can do this because I'm pretty sure that if I'm hired for an actual job, I'm able to do this. But I feel that it's frustrating to have all of this and like job requirements. And I feel that to compete with people who are confident that they, they would say, oh yeah, I know how to do this. For me, I don't want to say that I know how to do something if I don't.
You've hit on a very important, I mean, it's very difficult to write a good job description. I guess my thoughts on this are a couple of things. First, I'll hit on the last one you hit that I would rather somebody that I'm hiring tell me what you just said, which is, you know, I'm not going to bullshit you about my experience in this, but look at all the other things that I've done in the past that has been successful. And I have some basic knowledge in generative AI. I can learn and do things.
I don't want somebody being as basically as confident as the hallucinations in ChatGPT. And I want somebody to be honest with me. So when it comes to the job descriptions, you've got to meet some of the criteria that they put in there, but also realize that the person they hire is not going to meet all of those criteria. So, you know, if you think that you have most of the qualifications, go ahead and apply and also try to find somebody in that company. But, you know, if you really enjoy generative AI, then study it. But I will say there's, this isn't going to be the last big thing. So if this is not an area where you want to dive into the details, then know enough to get that role and do some things. But don't, you know, there will be other big things that happen in your long career in this.
Bridging statistics and machine learning
So this is related to Svetlana's question and kind of tying into the response, Warren. So like you, earlier on, I had, well, one of the defining points of my own career has been kind of helping folks bridge the gap between inferential statistics and machine learning, the unexplained stuff. And I'm actually finding that that area still holds a lot of interest for me, just intellectually and in some of the things that I like doing. So what are some of the things that you see maybe kind of moving forward happening in that field or in that area of data science that you think might kind of come about in the near or farther out future?
Yeah. So if you're talking about, I guess, generalizing it, not everything's going to be new. We still need to do a lot of the older, the more traditional things. And statistics is, is a, has a lot of great application because it's there to teach some things when we didn't have the amount of data that we have right now. I know at Best Buy, my team was fairly diverse and large, and some of them were traditional statisticians and some of them had come out of some of the more recent programs. And I'll say that when some of the more traditional statisticians would talk about uplift modeling or causal inference, two of those things are still super important. But they're getting a resurgence right now, especially the causal inference.
There were from the newer programs were like, yeah, I kind of like dismissive, like I read that in a textbook. Like, you know, it's not that important anymore, but when we actually applied it to some of our marketing campaigns and it made a significant difference. So I still think that if that's an area that you think you enjoy and then keep bridging that statistician to ML, like I'm talking about optimization to AI and ML, because I still think that in some of the areas you, machine learning starts with a lot of data and then you put some algorithms and it infers some models to it.
Things like integer programming starts with a model of the process. And there are so many things that we can do with a model of the process rather than starting from scratch each time. And in an example in reinforcement learning is you're teaching a car to drive itself. Then as a human, we implicitly know things like, hey, if I learned that I'm turning the steering wheel left and the car does this, then our model, our brain has this model of symmetry that says, basically I can do the same thing when I'm turning to the right. But just a standard reinforcement learning algorithm doesn't have that built in symmetry. So you can even put, infuse some models of gravitation or symmetry and things like that, even into reinforcement learning.
Most proud projects
So, you know, that's my thoughts on those is that you have a, find what you really enjoy doing. And that will be your niche. And hopefully it will also give you a long career as well.
Well, there's a few, you know, there's a couple, one is an optimization project at Lucent that the, so we've heard the famous saying, and I can't remember who said it, all models are wrong. Some models are useful. So there was this integer programming model that assigned raw materials to the cables. And at the time, these cables were made of stacks of ribbons of fiber, so they could be up to 864 fibers in a cable. And the pressure on the sides of a circular cable on this rectangular stack caused issues on the edges. And so we call them edge ribbons or corner ribbons. And for a while there, you wanted to, you didn't have enough of that really good fiber. So you had to do it a specific, you had to really allocate it the right way.
Then a few months later, well, maybe a year later, our factory processes had gotten so much better that we just had way too much of this. And we were trying to keep it separate. And then we started to not be able to give out orders or complete orders because we were taking this premium fiber and never using it. And some of us got together, got on a whiteboard and found out that there, if you took this substitution matrix of what ribbons could substitute for each other, like this one's better than this one, so it could substitute for the other one. And you put that matrix in there as a minimum and the transpose of that matrix as the maximum, then all of a sudden, just by adding those two matrices in there, everything could substitute for each other. And we were able to increase the capacity of the factory and reduce waste. And it was all because, you know, we got together and we used, you know, a couple of mathematical ideas.
And then the one where sometimes it's just a conversation or something that you read, there was at UPS, there was this idea that you used yeses and nos on the bids that you sent out to your clients, your big commercial clients. If they said you would give them, we'll give you 10% off retail rate and they'd say either yes or no. And you get tens of thousands of these data points and you create this logistic regression that can give you an idea of what your competitors are charging, because you think that everybody's rational. If UPS gives you 10% and they turn it down, then likely FedEx or somebody else gave them an 11 or 12.
Well, the results coming out were good, but they didn't make, you couldn't explain them. And it turns out that it was because of the data that was coming in. It was basically censored. You didn't ask them, I'll give you zero. And they say, no, I'll give you one. They say, no, I'll give you two. They say no. And then finally you get to 10. You got all these data points right around the answers. So without, now I'm realizing that I'm saying something that I'm not actually showing you on a graph, but make long story short, I read something in a paper where somebody called them shadow data points or mirror data points. And we infused the actual data set with we said, if you accept it at 10%, then you would have accepted at 11 or 12 or 13. And we put all of those in. And if you turn this down at 10%, then you would have turned us down at nine, eight, seven, six, five, four, three, two, one. And when we put all of these data points in there, everything worked and you could explain it.
And we actually patented that idea that you could infuse data sets in this particular application with what we call shadow data points. So sometimes it's, you know, you can augment your data set with successes and failures that weren't really there because you think that a rational person would, if they're going to accept at a certain price, they would certainly accept at a less price, or they're going to turn you down at a certain price, they're going to turn you down at a higher price. So that was a really interesting thing that worked for UPS. I saw somebody at a conference that they were in hotels. They described the same problem. And I said, since you're in hotels and you're not competing with UPS, here's our solution. And it worked for them as well.
Return on investment for data science
Yeah, so I would like to ask about the return on investment on all the technologies that you were using. So from my experience, not always, but quite a lot, we have the situation when we hope for, or we plan for certain investment, but we won't get as much as we want due to certain things popping up, right? Unexpectedly. So I wonder what are you measuring return on investment, planning that, and what is your experience here?
Oh, that's a great question. And as you go up from individual contributor to manager and then to executive or officer at a large company, that becomes an even harder question to answer in the sense that we, you know, we have, you're talking about technology, but we looked at it as, you know, we have resources, we have people. So I have had salary and things and benefits. We also had what we were doing in Google cloud. So, you know, what were the compute costs, the storage costs. But there were also a lot of data engineering that weren't, that wasn't in our data science group that we didn't really know exactly how much that was costing because we were having to move things from on-prem to into the cloud, this migration.
So when it comes to actual technology tools, we like the open source approaches. We know that you can get a lot more bang for your buck, you know, when it's either open source or you go to that next level. And then also when it comes to optimization, there are some pretty good, if you're talking about optimization, there are some pretty good open source optimization solvers out there now that until you have a problem that doesn't solve in a reasonable amount of time, then you can prove it out in a proof of concept and a pilot for your business. And then you can go to a larger solver company, like a Gurobi or something like that, where it's going to be more expensive, but you'll be able to prove it out.
The software costs were only part of it. It was the salary and benefits. It was also, what are you deprioritizing? I'll say at the executive level, it's the first thing you're doing is prioritizing the work of the enterprise. And there can be some really highly, I guess, return on investment, you know, 2x, 5x, 10x return on investment that you can't actually do. And if you're an individual contributor working on that project, you wonder what the heck, I mean, we could spend a million and get 20 million out of this and we're not doing it. But it's because there's a whole lot of things that are being negotiated at the prioritization level. So it's not always about the return on investment. It's a lot of times it's about what are we going to focus on this quarter, this half or this year? And what did our CEO tell the street that we were going to do?
Defining ML vs AI
Hey, Warren, thanks for all the insights on this. I guess I was starting to get more curious as you talk about, you know, ML, personalization, generative AI. Do you mind explaining the difference between ML and AI for prediction? You know, I imagine there's like different models and stuff like that, but what is, I guess, the difference simply put, and why are companies, I guess, investing so much in AI personalization?
Well, you know, there is a big debate as to what is AI, what is machine learning. And in the society, I'm on a board of a 12,000 member society called Institute for Operations Research and Management Science. So we do the integer programming and things like that. There was a debate, should we go from operations research to analytics, you know. So the way we were saying it at Best Buy, so I drafted up our AI strategy and what we were saying, and it may not match what you all are doing, but we didn't want the business to have to figure out, is this gen AI? Is this machine learning? Is this optimization? We basically took the easier route of saying our AI strategy, because that's what's popular right now. Anytime you're using complex math, math algorithms, and some data to make a decision, we're going to call that AI, and you need to come to at least brief the AI steering group on what project it is.
So, but my personal opinion is that AI in the traditional sense are things where a computer will act like a, more like a human. It can understand speech, vision, it can talk to you and interact like a human does. That's my traditional view of AI. Machine learning is any algorithm that gets better over time. ML in the 90s was a very specific set of algorithms like neural networks, reinforcement learning, some genetic algorithms, and things like that. Things like logistic regression and linear regression, those were statistics. They were not part of machine learning until the more the layman's term of machine learning became basically any algorithm like that.
And then things like optimization, I don't, that are more model based, I don't see those as AI in this traditional sense, but I will call them AI. You all use optimization every single day when you turn on Google maps and you use GPS. That's not machine learning doing that. Those are mathematical algorithms using, you know, some type of shortest path algorithm that you all probably learned in undergrad, you know, Dijkstra's and some other types of algorithms that are probably a lot more complicated than what you learned.
But so those are my personal definitions, but I would say for, if you're at your company, don't ask people to try to figure out what's generative AI and what's AI and what's ML. Have some central location because it's a little bit easier to at least have them come to the experts. Don't be the gatekeeper and say you can't do it, but at least say, come to us because in finance, you know, there's a lot of things that generative AI can do. Some of the really neat things about generative AI are that you can do some of that initial exploratory data analysis without having to write the code itself. It's like, you know, you put in some data and it's writing the Python code and does some of those types of things. And it can do some basic linear regression and logistic regression and some of the basic algorithms.
Pareto distribution of techniques and dormant ideas
I want to make sure I go over to where some of the anonymous questions were asked. And one over there was in your experience is the impact of data science techniques Pareto distributed. So is there 20% of techniques that drive 80% of the outcomes?
I would think so. Just for the sheer amount of techniques that are out there that didn't get traction for some reason, you know, I will, one of the things that I will be doing now that I'm semi-retired is I'm going to take my algorithms from the mid nineties and apply them to some of the open AI or, is it open AI gym? You know, some of the reinforcement learning out there. We didn't, we didn't open source. There's so many algorithms out there that are not open sourced from either PhDs or from companies. Some of the techniques out there gain traction and that 20% is doing 80% of the work. So I would say, yeah, there's just a whole lot of great ideas that we haven't really taken advantage of.
I'll add to that, just take, take for example, neural networks. The idea of a perceptron, I think was 1948 that they came up with this idea that, you know, we could do a single neuron, a perceptron. Then it was in, you know, kind of thought about different things. It was in, I think variations, but in the early seventies, a couple of people came up with the idea that you could do back propagation. Paul Werbos did it in his PhD thesis in 1974. And I know that because I invited Paul Werbos to a conference in 2001, when he was a director at the National Science Foundation. And even in 2001, the reason he wanted to come and talk to this conference was he had millions of dollars worth of NSF grant money that nobody was asking for in the area of neural networks.
So can you believe, I mean, nowadays everybody's asking for money and there's, you know, it's all taken, but he was basically begging everyone, all the researchers there, it's like, I have millions of dollars, just come ask for it. I will give it to you and you can do some research. So Paul Werbos came up with that in like 1974. He didn't actually get it published because, and I was checking the Wikipedia page earlier, people didn't really buy into that and he couldn't get that published until like 1982. And then some others came up with similar ideas in the mid eighties. So there were several people that came up with it, but that's decades worth of great ideas that nobody really acted on.
So can you imagine what great ideas are sitting dormant out there right now? And 10 years from now, somebody's going to look at something and say, yeah, that helps. Reinforcement learning with human feedback. I've never would have thought that reinforcement learning would have been one of the keys to get ChatGPT, to get that step change. You know, so somebody that I thought it was for robotics, you know, and here they're using it for natural language processing. And those of us that been studying this for years still didn't see that.
Reinforcement learning in practice and life
So I'll talk about it first at a high level. I got really interested in reinforcement learning because I've had a dog for most of my life and I just like to see how they learned and you could teach them to do tricks. And actually while I was a grad student, my dog knew probably 20 different commands. It knew everything from, you know, high five, to if you ask it, which would you rather be married or dead, the dog would turn around several times and fall over dead. But you taught the dog through incremental steps.
And so our life and, you know, everything is a sequential decision process. A lot of machine learning algorithms like reinforcement learning, genetic algorithms, neural networks, they all have this basis in how humans, animals, or even simple organisms learn and survive. So that's the basis for a lot of that. I think it's sometimes instructive because we focus on that to kind of turn that on its head and see what those algorithms can teach us about our life.
And so with reinforcement learning, you know, like I said, it's a, your life is a sequence of decisions, some good, some bad. We learn from those and we try to make a better decision in the future. There's a saying by Mark, attributed to Mark Twain, good judgment comes from experience and experience comes from bad judgment. And it's basically, you know, we, all of us on this call know that we are constantly learning from our decisions. We make a decision, we're in a state, we take an action out of all the possible actions we could take. We get some type of feedback from that, either instant or delayed, and we update our model of the world. That's reinforcement learning.
And the key to that is feedback. So another thing you can learn from this reinforcement learning algorithm and apply it to your life is feedback is crucial to us getting better. So, you know, make sure that you get and give timely feedback to your kids, to your pets, to your coworkers, to everyone. So feedback's important. I guess the last thing I'll say on this philosophical bent is failure is absolutely key to learning and improving. Reinforcement learning is not, I'm going to succeed every single time I do something. You actually, you know, when you're doing a robotic process in what's called set point regulation control, there's only one final destination that gets you that reward and you fail thousands of times more than you succeed. So failure is key to learning. And so don't be afraid to try something and fail at it.
I guess the last thing I'll say on this philosophical bent is failure is absolutely key to learning and improving. So failure is key to learning. And so don't be afraid to try something and fail at it.
And I guess the, what I would say for neural networks is, you know, we all know that the more connections you have in a neural network, the better it is, you know, and basically we're like that too. You know, it's the connections that you're making here, the connections that you make in your job. And I'm finding now at the end of my career, it's not necessarily the money that I made or the algorithms that I created. It's the connections that I've made with people that made me successful, but also are the things that I look back on and say, that's what's going to last.
But I would say for reinforcement learning, if you have any type of sequential decision-making process that you're doing at your work, and you can get some type of feedback from that, it doesn't have to be robotic control. It can be a process of, you know, anytime you can get a decision, you make a decision and then you can somehow generalize that decision based on something that's good or bad that happened. That is reinforcement learning. And there's a big surge over the past, say five to 10 years of, you can look it up as sequential decision-making processes, approximate dynamic programming. There's dozens of algorithms that are out there that can help, but just think of it as anytime you're making more than one decision and you can get some feedback on it.
Environmental impact of LLMs
Yeah, so have you, so are you familiar with Sasha Luciani? Yeah, no, she's a AI researcher and she gave a TED Talk about, you know, the environmental impact of ChatGPT. And her analogy was essentially that every time you type a knock-knock joke into ChatGPT, it's essentially the equivalent of driving your car around the globe. So I'm just wondering, do you have any opinion on that, on the environmental impact, on the amount of resources that are required to run these data centers, the amount of time it takes to run the data centers, the CO2 that's being produced?
I don't, you know, I do know this, as in the hype of generative AI, everybody saw 3.5 and then 4 and, you know, and we're going to have 5 pretty soon, and that's just with open AI. What I was telling the other executives at Best Buy is we're going to use this to see how, you know, like, what's the best that can happen. You know, you're going to use, let's say GPT-4 to summarize your call center and to figure out the intents and the topics and things like that. Well, my go-to saying was I don't need an LLM in a call center that can respond in a Shakespeare sonnet like Snoop Dogg would say. You know, it is overkill what these very, very large LLMs can do.
So what I think is going to happen is you're going to see a lot of companies really go into generative AI and then they're going to, just like with the cloud, they're going to say, holy crap, this is a lot more expensive than I ever thought. And back to the ROI, the ROI is not going to be there. So people will start to figure out what's the smallest model that I can do that can be. And so I think there's going to be this, you know, maybe at Hugging Face, something just for call centers or something just for, and it will reduce that environmental impact. But I think the hype right now is everybody wants to go into it and try it.
And then, you want to use like a hundred thousand tokens. Well, I don't want to pay for a hundred thousand tokens. I want to pay for 8,000 tokens or 16,000 tokens. I know I can get a better answer with a hundred thousand tokens, but it's probably not going to be worth it. And then lastly, there are some things where you'll see that I think they're going to pre-compute some of these. For some of the conferences that we're thinking about, instead of letting people use generative AI every single time they ask a question about some abstracts, we have like 6,000 abstract talks at one of these conferences. Let's pre-compute the abstracts. Let's say here's the original, very complex abstract. Now let's do it one time. Let's simplify it. The prompt is simplify this so that a first year's master's student in operations research or statistics would understand it. So I came up with, for I think $10, I came up with $10 worth of tokens for 6,000 abstracts, pre-computed all of these things. Now let's let Elasticsearch and do some things that are a little bit more reasonable. So I think that just capitalism will, for some of these companies, they will say, I can't afford to spend that much.
Career advice
Yeah, I've already talked about, you know, doing what you love, because if I had switched out of machine learning and gone to what was popular at the time, then I wouldn't have the career that I have. So, but I was fortunate that even though I said, people told me you're not going to find a job in what you're doing, that eventually I did. I guess the other is, especially for those of you that have masters or especially PhDs, early on in my career, I let the data do the talking for me. And we hear a lot of these days about data storytelling. And I think that's an important skill. You're going to have to make sure that you prove your value and the value of the models.
And basically every organization says they want to be data driven, but that's until the data actually goes against the gut feeling or instinct of the executive that's running the store operations or the executive that's running supply chain or something, then they don't necessarily want to be as data driven. Because think about it, back into reinforcement learning, they didn't get to be super successful executives by making bad decisions in the past. They've been rewarded by their gut decision is more often right than it is wrong. And all of a sudden you're telling them, well, this is not the best decision.
And so another plug for RStudio, one of the things that I did when I was at Cardlytics is, you know, the Shiny, using R Shiny was the best way that I could, that I could do to let somebody visualize, let an executive that's not analytics or data science visualize what either uncertainty or their assumptions about the process could actually do. And so, you know, that's, I would say, you know, learn how to incorporate that uncertainty or visualize it or do some what-if scenarios so that you're either going to learn that your model doesn't take everything into account and you're going to fix it, or you're going to get more buy-in from the other executives.
One of the things that I did when I was at Cardlytics is, you know, the Shiny, using R Shiny was the best way that I could do to let somebody visualize, let an executive that's not analytics or data science visualize what either uncertainty or their assumptions about the process could actually do.
Thank you so much, Warren, for taking the time to join us today and for sharing your experience with all of us. This has been a really fun, fun Hangout. Thank you all for the questions too. I always like to add this reminder at the end for anybody joining right now, just double check that you have the Hangout on your calendar for the right time. If you want to re-add it to your calendar, I'll put the link in the chat because it's always from 12 to 1 Eastern time.
But I did see a question about, if you decided to follow your idea of testing less common models, people would love to see the community effort around that. Is the best place to connect with you LinkedIn? LinkedIn is the best place right now. And then I'll come up with something else. But yeah, I'm always on LinkedIn. I love it. Okay, perfect. Well, thank you so much. Have a great rest of the day, everybody.