Data Science in People Analytics | Led by Elizabeth Esarove, AT&T
People are the face, heart, and hands of a company. In people analytics, we analyze data to reveal actionable insights that provide evidence for decisions regarding employees, work, and business objectives. This talk will cover the use of data science for people analytics projects such as workforce planning, improving employee engagement, and retaining talent. Speaker bio: Elizabeth Esarove is a data scientist in People Analytics at AT&T. In her role, Elizabeth is part of a larger team focused on embedding data and analytics into the root of decision-making and transforming insights into actionable solutions that improve employee outcomes and drive business value. Timestamps: *Q&A timestamps listed further below 3:42 - Start of session 5:14 - What is People Analytics 6:26 - Opportunities for Data Science in People Analytics 7:10 - Using Predictive Models to Reduce Attrition 11:10 - Segmenting Your Population 18:55 - Communicating with Leaders 20:11 - Time Series Forecasting for Workforce Changes 24:41 - Analyzing Employee Survey Comments Helpful Resources Below: *more follow-up to come with a Q&A blog post in the works People Analytics Books Mentioned today: Handbook of Regression Modeling in People Analytics: with examples in R, Python and Julia by Keith McNulty https://lnkd.in/eBFgniFG Excellence in People Analytics: How to Use Workforce Data to Create Business Value by Jonathan Ferrar and David Green https://a.co/d/bJrMRuW People analytics books shared in a previous data science hangout: Predictive HR Analytics: Mastering the HR Metric: https://a.co/d/5Hx05mw Inclusalytics - How Diversity, Equity and Inclusion Leaders Use Data to Drive Their Work: https://lnkd.in/g48tdrMu Other links shared by Liz: Time Series Models Forecasting: Principles and Practice by Rob Hyndman and George Athanasopoulos https://otexts.com/fpp3/ Text Analytics Text Mining with R by Julia Silge & David Robinson https://lnkd.in/emawveZd Additional resources shared: R Gov Conference: https://lnkd.in/ePfN7jru (David Meza is presenting on the RStudio (Posit) Ecosystem as a Critical Part of NASA Analytics Capabilities) People analytics for getting to the moon | Data Science Hangout with David Meza, NASA: https://lnkd.in/eDirbgCF For LATAM and Spanish Speaking people, Sergio Garcia Mora shared the R4HR community which has developed lots of free access content: https://data-4hr.com/ John Kelly IV shared the Human Resources Science LinkedIn Group: https://lnkd.in/eEMpYAfk Adrian M. Pérez shared the People Analytics Handbook: https://lnkd.in/ecsWy-dA Data Science Hangout: pos.it/dsh All upcoming #Posit community events: pos.it/community-events Q&A Timestamps: *the following timestamps are approximate. 16:00 - What are the most important people analytics KPIs @ AT&T? Can you share how your team/HR acts on these predictions (for optimal policy) both experimentally and ethically? do you implement new policy in smaller groups? 23:00 - How have you validated the predictive models? Looking backwards, how precise were they? 25:00 - Do you work with your HRBPs to segment your population? 25:00 - What languages are you using to build your predictive models? 31:00 - Do you include demographic information (gender, race, age) in your models? 31:00 - Are your surveys anonymous? 32:00 - How would you get the ROI from HR attrition modeling? 34:00 - Are most data scientists from a Psychometrics background? 35:00 - Is there a kind of "critical mass" to apply People Analytics? (just for big companies?) 36:00 - Looking at positive / negative comments, do you quote verbatim comments in your reports? (e.g. "here is one of the very positive / very negative comments we received") 37:00 - Do you use something like Snowflake to store and model your data? And do you deploy these models automatically or manually update them? 38:00 - R user here. How do you balance between people-ops focused analytics tools from outside vendors (often very expensive, but helpful) with custom in-house analytics (often time-consuming)? 41:00 - How much of your work is driven by HR leadership, by HR business leaders, or by the HR analytics team pushing modeling and insights to those groups? 42:00 - What was your journey into learning data science and getting into people analytics? 44:00 - Do you have a role in education business units? to improve their questions, etc.? 45:00 - What is the HR tech stack at AT&T? Does your team have a data engineer solely for people data since they're more sensitive? 47:00 - How do you present your results? (an application, report, power point) and how important is it to learn other languages (javascript, css, sql)? If you were to start a people analytics team in a company (+1000), how do you start? 50:00 - Do you use an internal tool for surveys? Do you use thresholds to maintain anonymity? 53:00 - Does AT&T have remote workers? If so, does people analytics segment on remote vs hybrid vs on-site?
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Weekend and happy Monday. Welcome to the Posit Enterprise community meetup. That is my first time saying that. In case you haven't heard yet, last Wednesday, RStudio became Posit. And I'll share that blog post on the announcement. It looks like there was more than our usual few second delay here. So let me restart all that. But if you just joined now, feel free to say hi through the chat window and maybe where you're calling in from. Special welcome to you if this is your first time joining us for this meetup today.
This is a friendly and open meetup environment for teams to share use cases, teach lessons learned and just meet each other and ask questions. These normally happen on Tuesdays at 12 Eastern time. Posit has a holiday tomorrow. So we moved it to today. But I will share a link in the chat where you can also find out about other upcoming events, too. Together, we're all dedicated to making this space inclusive and open for everyone, no matter your experience, industry or background. During the event, we'll take a few breaks for questions and have time for Q&A at the end, too.
Today, I am really looking forward to learning more about data science and people analytics. While I personally don't come from an HR or People Ops background, I have had the opportunity to hear a lot more about this recently at our Thursday data science Hangouts. And it's actually where we first met our speaker for today, Liz Esero. A few months ago on the data science Hangout with Dave Ameza at NASA, we were talking about people analytics to get to the moon. And there were a bunch of comments and interest about sharing more work done in the people analytics space and learning more about this space.
So here we are today. Thank you all so much for joining us today. I'm very excited to introduce our speaker, Elizabeth Esero. Elizabeth is a data scientist in people analytics at AT&T. In her role, Elizabeth is part of a larger team focused on embedding data and analytics into the route of decision making and transforming insights into actionable solutions that improve employee outcomes and drive business value.
And so with all that, thank you, Liz. Thank you for joining us, Liz. Can you hear me OK? Yes, I can. OK, perfect. I will head backstage, but I'll be in the background pulling together questions and I'll come jump back in here in a few minutes. Thanks, Liz. OK, and I will pause periodically for questions as we go along. So feel free to post them. And Rachel, feel free to interrupt me if you have some questions about a slide I'm displaying so that I can stop and answer that while it's on the screen.
Introduction to people analytics
And I thought I'd introduce myself a little and tell you something about my background here. I've been a data scientist at AT&T for about six years, focused on people analytics, partly due to my background in human resources. I've been in HR for 20 years in various analytics based roles. And several years ago, I started seeing that many companies are beginning to use predictive models in HR. So I went back to school to get a master's degree in data science and have been applying data science techniques at AT&T ever since.
We're going to start by talking a little bit about people analytics for those of you who may not be familiar with this field. And then we're going to go into three different areas where we found data science techniques to be very helpful for us in people analytics at AT&T. One of those is using predictive models to reduce employee attrition, using time series forecasting for workforce changes, and analyzing employee survey comments.
So to start with a little about people analytics, we started several years ago in AT&T, in HR analytics, where we were measuring a lot of the HR processes and techniques to monitor and improve those areas within the company. Then we expanded our data analysis into workforce analytics, where we are measuring patterns in our workforce data, and all in with the goal of informing decision making and improving performance. Now we've cast a wider net on the data that we're analyzing in people analytics, and where we're including not only workforce data, but we're also including business data, behavioral data, and outside research occasionally to help drive insights, improve performance, and solve business problems.
We found three areas of opportunity for data science in people analytics. One is regarding employee retention, where we've used predictive models to determine professional and personal characteristics associated with a high probability of leaving the company so that we can change policies and help improve retention. We've also found time series models to be very helpful in supplementing our strategic workforce planning needs. And like many companies, we have employee surveys where we ask employees to give us feedback about AT&T. And in those cases, we've found text analytics, more specifically topic modeling and sentiment analysis to be very helpful in summarizing thousands of comments that we receive from our employees.
Predictive models for employee attrition
One area of interest in many companies is how can we use predictive models to help reduce employee attrition? And if you're not familiar with the term attrition, that is a metric that we use in HR to measure the rate at which employees are leaving the company. We have three main turnover categories that we look at and which you also might want to look at at your company in terms of determining which areas you might want to focus on for your first attrition project. One is involuntary dismissals. The other could be voluntary separations or voluntary turnover that is related to retirement. And we could also have voluntary turnover that is not related to retirement where those employees, instead of leaving the workforce, they are leaving the company to go work somewhere else.
And we found it very helpful in looking at our attrition projects to also look at different employee segments and choose the ones that are appropriate for your company. But we found helpful to look at frontline employees who work directly with our customers, knowledge workers and executives or senior leadership. For each of these groups, then we consider the degree of difficulty of the project versus the potential benefit we may realize in terms of return on investment from achieved from reducing attrition.
For employees that voluntarily leave due to retirement, meaning they're leaving the workforce and aren't planning to go to work for another company, the degree of difficulty in reducing attrition is fairly low. However, the potential benefit is low to medium due to this being a smaller percentage of our employees.
However, when you're looking at voluntary, that's attrition that's not related to retirement. For frontline employees, the degree of difficulty is about medium. It's higher than those for retirements. But your return on investment is very high. With executives or senior leadership, the degree of difficulty is higher than those for frontline workers and your return on investment or potential benefit is lower, simply due to this being a smaller population of our workforce. And with knowledge workers, your degree of difficulty is higher than those for frontline employees. But your potential benefit is also just as high as you'll see with frontline employees.
For those that are involuntarily dismissed, your degree of difficulty is higher. And your potential benefit is low because this is typically a very small percentage of the population.
Attrition project roadmap
As you're continuing to plan an attrition project, we have a project roadmap that we found very helpful in consisting of four steps. Our overall goal is to identify the items that we can change and work with leadership in the business unit to help implement those changes with the overall objective of reducing attrition. So our first step is identifying those areas of the business unit where we have high turnover and also a good return on investment if we can reduce attrition. We then conducted research and developed predictive models to identify what those attrition drivers are, then developed interventions on the variables that we can impact or change in the business unit, and we selected certain locations where we implemented those changes. And then over time, we measured the impact to attrition for those locations to see how much impact we could achieve if we implemented them nationwide. And from there, we chose which items we wanted to change and roll them out throughout the entire country or throughout the entire business unit, whichever is applicable to your company.
In developing a project focused on attrition, it's also very helpful to segment your employee population to help control for any latent variables. And some areas that you might want to consider looking at could include employee tenure, where you're trying to figure out where is your higher attrition rates. Is that found among new employees or more established employees? You may look at management status, non-management versus management, employees, perhaps looking at supervisors versus individual contributors and see where you might have the most impact. Job type is a key one where grouping similar jobs together helps minimize the impact of latent variables. For example, our technicians who are working in the field, maintaining our network, are doing a very different job from, say, our sales associates and retail stores.
Doing separate attrition projects focused on each of these job groups separately helps us make sure we're getting good results from our predictive models to determine what are the attrition drivers for each of those job groups. And you might also consider looking at results of your employee engagement surveys to see how attitudes impact your willingness to stay. And just like job type, you might also consider full-time versus part-time, or if you have shift work at your company, consider that as well. Grouping those different types of employees together may also help improve any attrition project or predictive models that you are building.
Any questions before I move on? I'm seeing quite a few questions starting to come through Slido, and let me jump over. There were a few about KPIs, and one was what are the most important people analytics KPIs at AT&T? Oh, we have several, and depending on what are the needs of the business, we develop them, but the one focused here in this case for an attrition project is there is a standard attrition KPI that all companies measure in HR, and that is the number of employees that separate divided by your average headcount for a specific time period that you are measuring, and that will give you, you multiply that result by 100, and that will give you an attrition rate. We always express it as a percentage, and if you're looking at tenure groups, you may look at your number of employees who separated in that specific tenure group divided by your average headcount in that tenure group to get a more specific measurement for each of the tenure groups that you're looking at, for example.
Thank you. Another question, maybe I'll take three questions or so right here, but one was do you distinguish between attrition versus turnover based on the position being backfilled or not?
Most of the time we measure attrition in terms of whether it's involuntary or voluntary, meaning the employee chose to resign and are either going elsewhere or perhaps if they're voluntary and they're also retiring, and we know that for a fact, we might look at attrition just for those that are not retiring, but we don't measure it in terms of whether or not we're backfilling the position. In that, in the attrition calculation itself, we don't.
Great, thank you. There was one other question I see that just came in on YouTube, and I'll pull that up on the screen. But Paul asks, can you share how your team or HR acts on these predictions, both experimentally and ethically? Do you implement new policy in smaller groups? I will get to that on my future slides. Yes, our goal is to change policies to try and improve attrition.
Inputs for attrition predictive models
So some of you may be wondering what are some of the potential inputs that we might consider including in a predictive model that's focused on attrition. And we've grouped these here in terms of professional, personal attainment, attitude and other items that you might consider. And so some items and these aren't specific to AT&T. These could apply to any company. So you could include job titles. I mentioned grouping similar job titles together, department locations, full time versus part time, for example, among personal characteristics. If you have the data, you can consider education level of employees, their tenure, any information about their career path within the company, for example, commute distance for attainment. Look at performance information that you have. If you have customer satisfaction scores for specific jobs where that is measured, you could consider including that or any job specific performance metrics that are unique to the particular job that you are measuring for your current project. Under attitude, you could look at engagement survey results or onboarding surveys if you have those in your company. And for other items, you might even consider local unemployment rates to see how that might be impacting your attrition rates and supervisor performance metrics if you have those easily available.
There are also factors that we specifically don't model, and we work very closely with our legal counsel and our privacy team to determine what items are appropriate to include in our predictive models. And that could be from a standpoint of the legal perspective, as well as an ethics perspective to comply with AT&T standards regarding what we consider appropriate to include in our models.
So many of you may be wondering about the predictive modeling techniques, and I would encourage you to work closely with a data scientist. We consider this to be a partnership between HR and the data scientist. In my case, I'm a data scientist who's in HR, but I also work closely with HR business partners and business unit leaders to try and get a good understanding of what items they think might be affecting attrition as I'm developing input from my models. But then we go into the predictive modeling techniques. So if you are an HR person and you are working with a data scientist, this is where they will get more heavily into the technical side of things.
We've used several different types of machine learning models, and we test different ones to see which ones are the most accurate. We are specifically focusing in this type of case on classification models, where we are trying to predict a binary outcome of did the employee stay or did they leave based on all of the historical data that we've gathered of employees who stayed and left in the past and controlling for all of the things that are included in that model together in one place. We can then interpret the results of those models and determine which items are associated with a high probability of leading the company compared to other items in that model.
So interpretation of the model, for those of you who are data scientists, you may be familiar, some models are more easily interpreted than others. In people analytics, interpretation of the model is the most important.
In people analytics, interpretation of the model is the most important.
And here's an example of why it's most important, because when you are later communicating with business leaders on the results of that model, they want to know which variables are associated with high probability of leading, as well as which ones are associated with our ability to influence or change those items. And this type of visual is just one possible example of how you may present that data to leadership to let them know which areas that we might want to focus on where we're going to get a high return on investment for making those changes.
Time series forecasting for workforce planning
Another area where we found very helpful is forecasting workforce changes. And for this types of forecasting, we have been using time series predictive models. If you're not familiar with time series models, these are models where you are measuring something over a period of time. This graph on the right side of the page will give you an example, not specific to AT&T, but just to show you when you're looking at time series models, oftentimes you have to consider whether there's any seasonality in the data. So you can see on this example, there's an increase in value around the beginning of every year. And the predictive part of our model, which is shaded in blue, is also forecasting an increase at the beginning of each year. So this is the type of model that we have found very helpful on some cases for workforce planning.
One area that we've used this often is estimating the pandemic's impact on our workforce in 2020 and 2021. As each variant of the COVID virus spread throughout the U.S., we were able to make rough estimates of employee absences that we might see in different geographic areas. And that helped us ensure that we could do some planning, maybe even temporarily moving employees to other locations to help ensure that our customer needs continue to be met during the pandemic. We've also found changes like many companies, employee attrition rates have been very different in both 2020, 2021, even throughout this year. And we've been using some models to try and predict our future changes in employee attrition rates that we might see, which then fed into our strategic talent acquisition plans to try and determine how we needed to change or backfill positions as people left.
And our third area that we found this very helpful is in our future planning. As our plans for our product portfolio changes over the next few years, we've been using models to try and predict how our workforce may change in the future, both through voluntary separations or employees moving into other job titles in order to give us a feel for how we may plan for skills and job titles that may be needed in the future.
Are there any questions about time series models? Yes. Sorry, Liz. I didn't know I was muted before, but I was saying what I was saying was there's a lot of great questions, but I was trying to figure out which were most relevant to the section. But there was a question that is how have you validated the predictive models looking backwards? How precise were they? I don't remember exact numbers on how precise are they. I typically test in terms of time series models. I test two or three different algorithms and I choose we use if you're not familiar with predictive modeling, we divide our historical data into a training set and a test set and we will use one set to train the model and then we test the predictions on new data that has not been seen by the model yet. And that's where we measure the accuracy of those models. But I don't remember any specifics on how accurate they are.
Thank you. Another question a bit earlier was do you work and I'm guessing that this acronym is HR business partners. Do you work with your HR business partners to segment your population? Yes, we do. And we also if sometimes HR business partners will bring in the business unit leaders. And for those of you that aren't familiar, an HR business partner serves as a liaison between the business unit and HR. They are often the most informed about the current business needs of each of the business units that they work with. And so they've been very helpful in working closely with us to help us identify areas that we might focus on to improve the business.
Thank you for the definition there too. That's really helpful. I see one other question and then I'll let you get back to the slides here is what languages were used in building your predictive models? I build my models in R. I'm primarily an R programmer. We have other team members that build their models in Python. So we aren't exclusive to one particular language.
Analyzing employee survey comments
So our last area where we found data science to be very helpful is analyzing employee survey comments. And I mentioned in our surveys, we sometimes ask employees to give us feedback about AT&T where they're allowed to type anything that they would like to say. And when of course you're looking at thousands of comments, it's hard to summarize what are people talking about. So we found topic modeling to be very helpful in this area. The modeling approach that we use for those of you that are familiar with text analytics is latent Dirichlet allocation. And this allows us without going into the math and a lot of detail, this allows us to group together certain comments based on the keywords that employees use in those comments. We take out some of the common words like the and of, because every comment is going to have those words in them. And that then reduces down the comments to the unique words that exist in each of them. From there, we can then summarize those topics.
So here's an example of how topic modeling may work. And this is just based on industry data because we figure most companies are going to see comments about these topics. So you can see here on the left, I've got two lists of words here. And these are examples of comments, words that are coming from various comments from employees. And you can see there's two entirely different lists. Advancement, opportunity, career, promotion is entirely different from pay, retirement, salary, raises, insurance, and so forth. So from there, we can then decide our topic for the items on the left are more focused on career growth. And these items on the right are focused more on pay and benefits related comments. From there, we can continue this process to label all of our different topics that show up in the comments. And that then allows us to prepare some summaries for executives to let them know which areas are people commenting on the most, what are the biggest concerns of our employees. And from there, they can decide what actions they want to take to try and change things at AT&T based on the feedback that we've received.
Another area in text analytics that we found very helpful is we often get questions from management regarding the sentiment of those topics or the comments in general. Are they positive or negative? So in that case, we've been using lexicon-based algorithms. There are two general purpose lexicons available in both R and Python. One is AFIN and BING, and these are developed by researchers who have examined a lot of comments and determined that certain words are representing a positive or negative sentiment in those words or in those comments. However, for people analytics cases where employees are commenting about their jobs or awards or benefits, we found it helpful to customize these lexicons to give us a better understanding of the sentiment expressed in work-related comments or job-related comments.
So we've removed some words from the standard lexicons, and those words are listed here on the left. And you can see things like pay and work. They're more general topics of comments about pay and work, but what we really want to know is what is their sentiment about pay and work, for example. So we found it's easier to remove those words, and then we can get a better, more accurate understanding of the sentiment of their comments. And then you may find, looking at the comments in general, you may find there's words that should be added, maybe even some that are unique to your company or your industry, that would help give you an even better understanding of sentiment. And some words that don't exist in the general purpose lexicons, like burnout. We're seeing a lot of articles about employees feeling burnout these days. Micromanage, which seems to exist in almost any company. These are things that you may consider adding to your custom sentiment lexicon to help give you a better understanding of your sentiment in these employee-based comments.
What we do overall in sentiment analysis is then from that, once you've customized your lexicon, we apply that to the comments and try and measure the number of positive words and the number of negative words that appear in each comment. And we take the positive words minus the negative words and give us a general comment score so that we can see, overall, is that comment mostly positive or is it mostly negative? And from there, you can give management an understanding of, are the comments about a specific topic more positive or negative? Or even just overall, among all of the comments that employees make, are they mostly positive or negative?
Q&A
There are a lot of great questions. One question was, Heather asked, do you use demographics like age, ethnicity, and gender to predict attrition? We work closely with our legal and privacy team to determine whether it's appropriate and what cases it might be appropriate to include those. So I would recommend that you also do the same at your company before putting them in your model.
And another question was, are your survey results attributed to employees or are they anonymous? They're anonymous, but we have the ability to try and group those data so that we know, in general, for different types of groups of employees, what are the results that we are seeing.
Thank you. And another question, which I think also touches on when you were talking about presenting to leadership, but how would you get the ROI from HR attrition modeling? We work closely with our finance department to determine the cost of attrition or the cost of replacing a new employee. And from there, we can then determine how much would it cost to replace that employee based on our attrition or how much we would, how much savings we would realize if we can reduce the rate at which we need to replace employees.
Great, thank you. And I'll save some more of the questions for the end here. But judging by all the great questions, it seems like we should continue having many more people analytics chats too. Well, actually, that is the end of my presentation. So if we have other questions, feel free to let me know. Great, thank you so much, Liz. Yes, there are tons of questions here.
A few questions were kind of centered around like how to get into HR attrition modeling and how to get into like how to get into people analytics and maybe what kinds of backgrounds people might come from. One question was, are most of the data scientists there from a psychometrics background, or are there specific classes or courses you'd recommend?
Oh, boy, that's, there's a lot of different ways we could go. We, on our team, since data analysis skills are extremely important, we've had people who joined us that had data analytics skills from other parts of the company, we tend to hire internally or post our job openings internally before going outside of AT&T. So that's why I mentioned the people that have joined us either have data analytics skills from other parts of the company, and they come into the people analytics team, or they have a background in HR with strong data analysis skills, and they've decided to specialize in people analytics, which is the path that I took. And we also have data scientists who come from other parts of the company and decided to come join us and specialize more in people analytics.
As far as learning the people analytics side, there are some courses on LinkedIn learning to help get you familiar with the general field of people analytics and some of the HR metrics that are commonly measured. And I there is an excellent book, Rachel, I think I gave you information about excellence in people analytics, which is recently published, and it will give you a very good overview to help you get familiar with people analytics. There's lots of example use cases in that book to help you understand different areas that other companies have use people analytics and data science techniques, even predictive modeling to help improve the business.
Awesome. Yeah. And I put I put that book recommendation into the chat. So I really I love this question, too. Is is there a kind of critical mass to apply people analytics? Is it just for big companies? Or do you see a lot of small companies with people analytics teams, too? I think small companies can use people analytics, too. They may find that predictive modeling might work better if you have a large employee data set. But there's a lot that you can do with statistical analysis. And if in fact, there's a book by Keith McNulty that provides several examples of other ways that employees could or people analytics teams could apply regression modeling and other statistical techniques to help solve business problems, and that can apply at a large company or a small company.
Lots of great questions here. So let's let's see. Alex asked on YouTube, do neural network modeling methods have a place in people analytics? I have not used them much. However, we're open to trying anything if if it works and you can interpret the results.
Thank you. And in what you are just showing on like the positive and negative comments, do you quote verbatim comments in your reports, for example, like here's one of the very positive or very negative comments we receive? Occasionally, if it helps when we're presenting results to leadership, sometimes it helps them to see more specifics of the comments that employees are sharing to get a feel for what are they saying about a specific topic.
That that slide there was making me think about like the difference between what someone's text might be in an anonymous setting versus in like a company meeting. And if there's like a different threshold for like negativity versus positivity in those settings. Like, do you find that people are generally more negative if something's anonymous? I don't know that we've measured what they say in a meeting environment where everybody knows everybody in order to be able to have some solid data help answer that question.
Thank you. I just want to jump over to some of the questions on Slido. And somebody asked, do you use something like Snowflake to store and model your data? And do you deploy these models automatically or manually update them? We haven't deployed any models in production yet, but we're working towards that. But AT&T does use Snowflake in some parts of the business.
One question I saw earlier was, our user here, how do you balance between PeopleOps focused analytics tools, often very expensive, but helpful with custom analytics, often time consuming? And I'm not sure I understand the difference between the two. Do you mean custom analytics versus tools that are developed by vendors? It's an anonymous question, but I'm going to try and guess. I think it's, yeah, how do you balance between maybe like proprietary software tools or expensive like PeopleOps tools versus like probably code first or open source data science with R or Python?
In many cases, we start with a cost benefit analysis and determine how much would it cost to buy that tool from a vendor, for example, versus developing it in-house ourselves. For AT&T, we may be a unique company because we already have a lot of employees with excellent software development tools or skills. And we already have data scientists working throughout the company. So developing these items in-house, sometimes it's more cost effective for us, but we also have some cases where we found it's better to work with a vendor. So it just depends on the specific situation. And I think that would be the same for most companies is take a look at your cost benefit analysis of what it would cost to do you have the employees to that have the skills that you need to develop in-house tools. And if you don't have those employees, what would it take for you to hire them versus what would it take for you to hire a vendor to have build something for you?
Thank you. I see Heather had asked on Slido, how much of your work is driven by HR leadership or HR business leaders or by the HR analytics team pushing modeling and insights to those groups? That's I would say the majority of our work, since we are designed around trying to solve business problems, it's usually based on requests that are coming to us from the business unit leaders. They identify an area where they would like us to do some analysis to help answer questions for them or help solve a problem for them. And so I would say that most of my work, I don't have an exact percentage, but most of it's driven by the business units and trying to solve problems for them or provide something of value to them. We have very few cases where we design the project ourselves and then go to the business unit.
I had a question for people who I'd be like just thinking about getting into people analytics now and maybe work on a people ops team. What was your journey into learning data science and getting into people analytics? My particular journey, I started in HR many years ago, based on the fact that I had data analysis skills at another company and I was not working in HR. So they hired me due to my data analytics skills, which back then was working primarily with databases and building reports and using tools such as I think crystal reports was one back then. And I've continued staying in HR because I've always found the work fascinating as our work evolved in terms of more and more advanced analytics that we were capable of doing here at AT&T. Then several years ago, I decided to go back to school and get a data science degree when I started seeing articles of how other companies were using predictive analytics to help solve business problems. And we decided we wanted to do the same thing at AT&T.
We had some other team members that were already starting to work in that area. And I just decided that would be my area of specialty. In other cases, we've had IO psychologists who have worked with our team, and they come from the psychology background, but they also have a lot of data analysis skills that they're bringing to the team as well. And occasionally we've had data scientists who have experience in other parts of the company who have joined our team and build up their skills and understanding of people analytics.
Hey, thank you. I also just wanted to say to everybody listening in, I know there's like there's so many awesome questions here. So if you are working in HR analytics or people analytics, and you're part of different communities centered around this, I'd love to have you share them in the chat as well, if you want to share them with other people here, or other upcoming events to I think it'd just be helpful for us all to share resources with each other too.
I see John had asked on YouTube, do you have a role in educating business units as well to help improve their questions, for example? Yes, on our team, we have consultants that work closely with the HR business partners and the business unit leaders at the start of a project when we come that there's something that they have in mind that they would like the people analytics teams assistance with. And they're so at the start of the project, we often clearly meet with them to try and clearly define what exactly are the business questions that they want to answer, what things may be feasible for us to measure and help them solve and try and get more definition or scope of that project.
Thank you, Liz for letting me throw all these questions at you here. Maybe we'll do five more questions. Does that sound okay? Sure. Okay. Somebody had asked, and I see it's one of the most upvoted questions now on Slido. What is the HR tech stack at AT&T? Does your team have a data engineer solely for people data since it's more sensitive?
We have database administrators are some of the roles that we have. And I guess that would also be considered data engineers. And in some companies, we also have software engineers who develop some proprietary tools for AT&T to assist us with reporting analytics. And I mentioned before, we have consultants who work closely with our HR business partners and business unit leaders to help define the projects and answer some of those early questions that we get are in the start of a project. And those consultants continue working with us throughout the project. So they may be meeting with us when we're presenting the results as well. And advanced analytics team members who are doing advanced reporting. We also have a products team who's developing reports and various tools such as Power BI and Power Apps. Whatever those tools are the best ones that fit the need for the business. Those are the ones that we choose to develop, whether it's something proprietary that's developed in-house, or if we can use something that's already existing, such as Power BI, we may develop reports in those tools. And then, of course, we have data scientists who can do our more advanced analytics projects, predictive modeling, text analytics, and so forth, and project managers to help us manage all of these projects.
Great, thank you. While everybody's here and so engaged, I wanted to ask as well a poll in the Slido. So I just asked over there quickly, if we were to have another people analytics meetup, is there a specific topic that we covered today or didn't that you'd like to dive deeper into and would really appreciate your feedback and thoughts over there too?
But diving into some of the other questions here, I see Angel asks, how do you present your results? Is it in application, reports, or PowerPoint? How important is it to learn other languages like JavaScript, CSS, or SQL? Presenting results can vary depending on the situation. So it could be in PowerPoint, on various reports, or internally within the applications, some of the proprietary applications that AT&T has developed. And learning other languages, I would say SQL is commonly used anytime we're pulling data from our databases. So everybody on our team is learning SQL or already knows SQL. I'm not sure if we have people that have skills in JavaScript or CSS. Some of our software developers, of course, are using specific languages for software development, but I don't know what those languages are.
Another question I saw, let's see, from Nivaldo was, if you were to start a people analytics team right now in a company of maybe 1,000 people, how would you start? Oh, that could take a very long time. Maybe that's a whole separate talk. Yeah, that could be a whole separate presentation. I would recommend the excellence in people analytics book has some good advice for teams that are just getting started. And also, you may find the authors are available on LinkedIn. They post blogs on a regular basis, giving recommendations for new leaders. And those may be the best places to start unless we have a separate presentation on how to start a people analytics team.
Okay. One question was around internal tools for surveys. Do you use an internal tool for surveys or do you use thresholds to maintain anonymity? Depending on the survey, some of them are managed by vendors to help keep that anonymity for the employees and some are developed internally within AT&T. So it's a mix of both for us.
Oh, I did want to announce also that one upcoming people analytics talk I was just thinking about is David Meza will be presenting on NASA's use of analytics in people analytics at the R government event in DC this December. And I actually have a 20% off code that Lander Analytics shared with me to share with our community. So if you are interested in attending that conference, let me know.
Okay. One last question, Liz, and we can ask this to everybody here as well to share is do you have any favorite R packages for people analytics? I have many favorite R packages. And I think it may not be unique to people analytics, but the player is probably one of the ones I use the most just for rearranging my data and doing exploratory data analysis. Oh, I wish I can remember. There are a few that I have recently learned about that are also very good for exploratory data analysis to summarize the data in different ways. And I cannot remember the name of them. But I also use a lot of the packages that are in the tidy verse. And then for text analytics, there are separate packages that are specific to the tidy verse, but kind of applying the tidy verse concepts to text. And they are mentioned in a book on by Julia Silge. And I can't remember the name of the other offer. But it's all about using the tidy verse for text analytics. Rachel, I think I gave you a link to that one. And it's available for free on the web if anybody is interested in getting into text analytics.
Okay, I'm sorry, I said one last question. But there's so many great questions. One more. Okay. Does AT&T have remote workers? If so, does people analytics segment on remote verse hybrid verse on site? Good question. And one that's coming up quite often in in many companies are considering remote workers. So yes, we do have remote workers. I for one am a remote worker. We had remote workers prior to COVID. And for those of you that aren't familiar with HR, the whole concept of remote versus office has become a big debate after COVID as many companies are discussing whether or not to have employees return to the office full time, or have a hybrid schedule. So we have changed our policies, we now have full time remote workers, we also have hybrid workers, and we have employees who work on site. Some of that is determined by the type of job they do. Obviously, technicians working remote is going to be difficult. And some of it just depends on the employee's personal preference. There are some people who prefer to be in the office, and like that environment working in person with others, and some who prefer working remote. It varies throughout the company.
Good question. And one that's coming up quite often in in many companies are considering remote workers. So yes, we do have remote workers. We had remote workers prior to COVID. And for those of you that aren't familiar with HR, the whole concept of remote versus office has become a big debate after COVID as many companies are discussing whether or not to have employees return to the office full time, or have a hybrid schedule.
Thank you so much, Liz, for sharing your experience with us, but also answering so many awesome questions shared here by everybody. I know I, unfortunately, I don't think I got to every question here. So I want to do my best to look at all the questions, gather them, and maybe we come up with some sort of like, follow up blog posts and get more of the people analytics community together as well. Thank you so much, Liz. Sounds like we have a lot of interest in this topic. A ton of interest. Yes.
And I also wanted to extend an invite to everybody here today. If you haven't been to data science Hangout before, we have those every Thursday, at noon Eastern time. And we feature a different data science leader from the community in a very informal setting to answer questions that you all have, whether that's maybe how somebody is using people analytics, or how they're thinking about hiring and growing their team, all the questions are up to you. We'd love to see you in that space too. That's actually where Liz and I met and got to talk a bit more about this topic. So I'm going to put the link in the chat right now if you want to add that to your calendar too. Thank you again, Liz, really appreciate your time today and in all your insights. Thank you all for the great questions too. Yes, thank you. And thank you, Rachel, for inviting me and thank you everybody for taking time out of your schedule to join us today. Absolutely. And I will end with what john said here. There's a real HR analytics revolution coming. Thank you all again. Have a great rest of the day.
