Data Science Hangout | Adam Bly, System | Decentralizing decision making
We were joined by Adam Bly, Founder & CEO at System. Adam asks, why haven't we seen it all connected yet? Primitives for leaders: (12:12) Adam shared how he learned from Daniel Ek, the founder of Spotify - the importance of decentralizing as much of the decision making as possible to increase speed, autonomy, and ownership regardless of the size of a team. This puts a great deal of emphasis on the primitives that you feel are important for every leader in the company and then for their teams to really embrace that, understand and make sure people really get it. This empowers teams to be agile, to be squads, and go off and do their thing. We put a great deal of emphasis at System on a set of first principles that govern how we think about the product and technology, and then tensions that shape any major decision. We put a great deal of emphasis on our values and talk a lot about those and use those as primitives when we’re making product decisions or technology decisions. We also put a great deal of emphasis on the use of data broadly in the company to inform our decision making. By really equipping everybody with those three things that we talk about from day one onward through professional development, 1:1s, and everything at System, it allows for a high degree of ownership, autonomy, and distributed decision making around the company. Where to find more? ► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu ► Data Science Hangout site: rstudio.com/data-science-hangout ► Add the Data Science Hangout to your calendar: rstd.io/datasciencehangout Follow Us Here: Website: https://www.rstudio.com LinkedIn:https://www.linkedin.com/company/rstudio Twitter: https://twitter.com/rstudio
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hi, everybody, and welcome to the Data Science Hangout. If you're joining for the first time today, I'm Rachel. It's great to meet you. If this is your first Hangout, this is an open space for the whole data science community to connect and chat about data science leadership, questions you're facing and what's going on in the world of data science. The sessions are recorded and shared to the RStudio YouTube as well as the Data Science Hangout site, which Tyler or someone from the RStudio team can share in the chat as well. But you can always go back and rewatch or find helpful resources. And we do also have a LinkedIn group. So if you ever want to continue a conversation and meet with people, you can connect there too.
But together, we're all dedicated to making this a welcoming environment for everyone. So we love when everybody can participate and we can hear from everyone. So with that said, there's three ways that you can ask questions today. You can jump in the conversation by raising your hand on Zoom, and I can call you in. You can also put questions in the Zoom chat. And feel free to just put a little star next to it if you want me to read it out loud instead, maybe a dog's barking or you're in a coffee shop or something. But we also have a Slido link where you can ask questions anonymously too. I think Tyler's sharing that in the chat right this second. Just want to reiterate, we love to hear from everybody, no matter your level of experience or the area or industry of your work as well.
But for today, I'm so excited to be joined by my co-host, Adam Blyde, founder and CEO at System. And Adam, I'd love to kick things off by maybe having you introduce yourself and sharing a little bit about your work. Sure. Thanks so much for having me. It's a pleasure to be here and I look forward to a great conversation.
Yeah, so I founded System a few years ago, and we recently launched to the public just a few months ago, actually, after a few years of R&D. I'm a scientist by background, started life doing genetics and biochemistry, grew up in Montreal, and kind of quickly got excited about not just the application of science itself and the pursuit of science, all things of that from sort of wet lab to computational, but also in sort of just the general application of scientific thinking and scientific methods to problems that on their surface don't seem scientific. And so it's clear that we want to apply scientific methods and scientific thinking to things biomedical and things environmental, but maybe not to how we think about global development or how we think about poverty or how we think about food security. And I think the more we can advance the tools of scientific thinking and the scientific method to address those kinds of problems, the better the world would be. And so that's kind of the central thesis that motivates my work and my career.
Yeah, so super briefly, most recently after founding Seed Scientific, a data science company several years ago, where we worked with organizations like the United Nations and governments around the world to apply data science and machine learning in the very early days, very, very early phase of the data revolution, when our team was made up of scientists and engineers, who today we would call data scientists and data engineers, machine learning engineers, started working on a host of applications of data science and machine learning to optimizing polio vaccine distribution around the world, advancing the sustainable development agenda at the UN, all the way through to building the recommender system for Beats Music when they were the headphones trying to go up against Spotify. And that led to then selling the company to the company being acquired by Spotify.
And I had the sort of great pleasure, privilege of joining an incredible team at Spotify, an incredible leadership team, and figuring out how do we go from what was at the time a pretty small kind of analytics function, maybe 20, 30 people, into what it is today, sort of several hundred people around the world, distributed into every part of Spotify's business, using data to drive Spotify's mission in the world. I left Spotify a few years ago to start System, and yeah, super briefly there, happy to dump into this further.
System's mission and vision
Our mission at System is to relate everything to help the world see and solve anything as a system. The motivation behind System is a further extension of sort of this lifelong belief that if we can advance the tools of scientific thinking to problems that aren't on their surface scientific, all things will be improved in the world. To me, when I think about sort of the biggest issues in the world today, we're in the midst of a pandemic, we see emerging epidemics confronting us today, we're facing the real implications of climate change around the world, we're seeing the challenges of systemic racism, particularly in the United States, we're seeing the impact of geopolitics in Russia and Ukraine, not only of course affecting populations there but affecting food security and geopolitics around the world. All these important issues are highly systemic, they're highly interconnected, they're highly interdependent, and so if you're a scientist, they exhibit properties of complex systems, and I'm a student of complexity science and have long been fascinated by the application of complexity science and broadly speaking systems thinking to understanding the nature of the problems in the world.
And our mission is really to recognize that the way we organize data and information in the world today, is largely based on language. So when you search for information today, you get back things that semantically match what it is your query is all about. But language doesn't know systems, language doesn't know the relationship between climate change and migration and food security. And we believe that by understanding how things are related, by elevating the relationships between things, and those being statistically based relationships, we can drive a more holistic understanding of problems in the world, of problems we face in business, of problems we face when we get a blood test result, really anywhere that we get data, we feel it's lacking context and the system around it. And so what we're really trying to do is build a technology that brings context, that brings that holistic view to any data point that we're looking at at any given moment supported.
But language doesn't know systems, language doesn't know the relationship between climate change and migration and food security.
The final point is that we're a public benefit corporation, and our mission to relate everything, to help the world see and solve anything as a system is really what motivates us and our values as a company. System is free and open, it's available under a Creative Commons license. We recently had the sort of honor of introducing and bringing the former CEO of Wikipedia, Catherine Marr, onto our board of directors. And so that's the long-term direction of System.com as our core platform is really something that will be free and open and accessible to researchers, professionals and the general public around the world.
That's awesome. Thank you so much, Adam. And thanks for describing systems thinking a little bit too. So I think so many of us here are probably systems thinkers, but maybe not necessarily calling it that or having the name for it. But I want to start, while we're waiting for questions to come in from everyone, I want to start asking this to people every time so we can get to know you a little bit better too. I'm curious, what's something that you like to do in your free time or outside of work as well?
I love to cook. That's sort of my other kind of passion in life. So I spend a good amount of time over the last couple of years, especially during the pandemic, outside of, I live in New York, so outside of the city. And it's been great to grow vegetables and be able to go out and pick stuff and make dinner. So I'm enjoying learning to garden and really love to cook.
Growing teams and decentralizing decision making
Yeah, that's great. So I get to ask the first questions before people start typing them in here. So I'm curious, you came from Spotify and you mentioned growing a team from 20 to hundreds and then moving towards your own startup. What has it been like working with such different sizes of teams?
Yeah, my company Seed Scientific, before Spotify that we sold to Spotify, we were about 50 people or so, 40-50 people. And yeah, my team at Spotify when I left was 300 or something like that. And now we're a 20-person startup, something like that. Honestly, there's so many upsides to both. It's incredible to be able to really start from first principles when you're building a team as we've done at System and emphasize values that are native to our culture and our mission. And then build a team that advances those values. And we can really think about all of that stuff foundationally in how we hire and how we design our ways of working.
At the same time, our ambition for System is quite large. And we'd love to have 2000 people today to be able to work on these problems. And so sometimes I wish I could take the same organization at Spotify and be able to use that full, incredible brainpower and tech power to be able to work on our mission. And hopefully we'll get there. Spotify was an incredible challenge to go from 30 people to 300 people over two years. So just incredible scale of how do you hire that many people? How do you train that many people? We reached a point because we were hiring globally that there's simply, there's just not enough people that we at the time, this was three, four years ago or five years ago, we started that process could hire. And so we actually invested quite considerably in our own internal university to train engineers and marketers and product managers and designers who might've had the quantitative methodological kind of instincts, but hadn't been trained in data science. And so we thought it was going to be really important to build those kinds of education programs that continues today, sort of incredible team working on professional development and sort of growing data scientists at Spotify.
Yeah. I think at the same time, like what unites it is I really believe in trying to identify what are the kind of primitives that allow teams, Spotify is a very strong, agile belief. And that really means kind of decentralization, federation of as much decision-making as possible to increase speed and autonomy and ownership. And I really believe in that regardless of sort of size of team. And so that means, I think as a leader that you, I learned from Daniel Ack from the founder of Spotify, observing him do that, that it really puts a great deal of emphasis on what are the primitives that you feel are important for every leader in the company and then for their teams to really embrace and understand and really make sure people really get so that then you can empower teams to be agile, to be squads, to go off and do their thing. And so we put a great deal of emphasis at system on a set of first principles that govern how we think about the product and our technology and the tensions that shape any major decision. We put a great deal of emphasis on our values and talk a lot about those and use those as primitives when we're making product decisions or technology decisions. And we put a great deal of emphasis on sort of the use of data broadly in the company to inform our decision-making. And so by really equipping everybody with those three things, and we talk about that truly from day one onward through professional development and one-on-ones and everything at system, it allows for I think a high degree of ownership and autonomy and sort of distributed decision-making around the company. And that's something I learned applying now again at another startup, but from seeing that manifest at a several thousand person technology company that really valued that kind of thing.
It really puts a great deal of emphasis on what are the primitives that you feel are important for every leader in the company and then for their teams to really embrace and understand and really make sure people really get so that then you can empower teams to be agile, to be squads, to go off and do their thing.
How System works
So I was just interested to know what kinds of tools are you using to build these systems models or estimate parameters and how do you evaluate the kind of combined models of all these different parts against the outcomes that you're interested in?
Yeah. I'll give you the couple of minute answer and then all of our documentation, all of our technical documentation is up on our website and would encourage, if you have interest to get more insight into the science and technology behind it, please read that. And our data science team has done a great job to communicate that, I think.
So system is a platform that starts from identifying sources of raw statistical evidence in the world. And that evidence today exists in a number of different places. For us, we've identified three in particular. Those are the corpus of scientific literature in the world. So papers that get published every day in every part of science through the great work of the research community and funding agencies to advance scholarship. The second is the advancement of open data and real world evidence that sits in data warehouses and portals and so forth, measuring things like COVID vaccination rates and vaccine hesitancy rates and so on and so forth. And World Bank figures and IMF figures and so on. And then the third is a growing volume of machine learning models out in the world predicting a whole host of outcomes from housing prices to food security and so forth.
And so we've built software to identify evidence of a statistical association in any of those raw materials or the potential to compute one. So from a scientific paper, it's a retrieval project. From a data set, it's a computation project. It's time series analysis. It's actually running correlation analyses on data. From machine learning models, we've built R and Python packages that allow our teams today, and we'll eventually open this up, to compute the importance of features in predicting an outcome in a test set. And then we'll look at that as well, that feature importance as yet another input into the full set of inflows coming into systems. So we've got all these inflows coming in. Then we have a huge job of normalization and entity resolution. So yes, we recognize dozens and dozens and dozens and dozens of types of statistical association from odds ratios to RCT experiments to SHAP values to so on and so forth to correlation coefficients and all these things and the metadata around them. So the strength of that relationship, the controls that were put on that experiment, the sign, the direction, all sorts of attributes of that relationship we retrieve.
And we have to make sense of being able to relate one data point, one piece of evidence to another. So we do all that work. And then we have to resolve on the name level that obesity is an important topic that our users want to be able to search for and understand what's the relationship between obesity and diabetes. But because there's lots of different ways, metrics, variables of measuring obesity, body mass index, body mass index over 30, patient has declared themselves self-identified as obese, so on and so forth. All of those then need to be grounded within what we call the topic of obesity. So then that's the next part of our platform. Then all of this gets organized in a large-scale graph database. And then the way we traverse that graph database when a user searches system is not based on language, as I was saying. So if the core architecture, if the core information architecture of search and discovery today is based on language, right? So you search Google for housing prices, Google's knowledge graph will relate housing prices semantically to home prices and real estate prices, and then rank that and give you the New York Times article at the top. System is saying, cool, we need to get to housing prices, but then what we're really interested in is in all of the factors that system knows one degree, two degrees, and so forth away are being impacting housing prices or are being affected by it. And then we assemble all of that, visualize that in a graph, and present all of the evidence alongside that for the user. So that's the core proposition. And then we're building applications on top of that. But that's what you can find today at a very high level kind of how things work.
Leadership, values, and culture
Hmm, I think a lot about the values and culture of the company. Right. So at Spotify, I, you know, was an active participant in helping advance our values and advance our culture and hopefully, you know, positively contribute to it. But I inherited those values from a founder and a founding team. And those were his and their, you know, decisions about what's what is important here. And what what really unifies us and what will be the lens through which we will make the hardest decisions, hopefully. At system, I spent an incredible amount of my time thinking about those values initially, and now really making sure that I'm equipping our team and our leadership team with the tools and the case studies from around the company for how to apply those things very practically.
So I'll give you an example. Openness is a core value of ours. And we believe as is as is positive impact in the world. And we believe that a big problem with the way technology companies, especially those of us who traffic in data and AI and historically haven't been scrutinized the same way that defense companies or, you know, energy companies might have been scrutinized by the public. I'm really encouraged by, you know, the general direction of our field towards greater sense of accountability for the bias in our work for the impact of our work for the unintended consequences of our work. But I think this, you know, it starts with establishing that as a culture, it continues in actually making sure it's practiced. But then one of the things we're trying to do, and we were very open, we sort of published a blog post about this a couple months ago, is our VP of product and engineering and our VP of a team we call global issues, kind of working on systems application to world problems like global health, where we're doing work with the Gates Foundation, published a blog post on something we're calling release risks. So tech companies are used to publishing release notes, alongside all their major releases. And, you know, so here's great features we've just released. And our idea was we wanted to publish alongside our first major release, which was a couple months ago, all the things we see that could be unintended consequences of putting this out in the world as a release. And so we laid those out after a process we ran internally, and then went through point by point how we have set out to mitigate those potential consequences, where we still see risk, how we plan to improve on that, and then, you know, hold us to account on what we plan to show you by the next major release. That's the kind of thing that really stemmed directly from our values into an action that then our leadership team, you know, took forward as a practice of our value. So that's the kind of thing I spent a lot of my time thinking about, especially given the mission and the nature of our company as a PBC, that I just wasn't able to given not being a founder at Spotify.
Starting a company and accessing networks
Yeah, there's been a lot of chatter I've seen on Twitter, LinkedIn, among our data science peers about, you know, when is the right time to kind of start a business, make a startup? How does, like, what are the first steps one would take, get seed funding, those sorts of things? And intriguingly, you came from a large, successful org like Spotify, and were able to spin off something successful now, a system. There's really good talk at RStudio Conf from David Kyes about what they forgot to teach you about starting a business. So, I mean, yeah, there's a lot of interest in this, and I'm curious myself, like, you know, when you know the time is right, what do you do? What's the next steps? Do you know, like, angel investors? Is it like a network thing? Is it something else? What does that look like?
Yeah, I think it's a great question. And I think it's awesome that so many, especially over the last, like, couple of years, have been feeling the motivation to sort of break out, to start new things and see a problem and just go off and pursue it. And I think that's awesome. I mean, I believe in that. I sort of try to help as many founders as I can. I have a general rule. I actually think, like, all founders kind of believe this is, you know, if another founder reaches out to you for help, you just, you help, like, you take the call. So, I do this a lot. Like, I'll just, like, go have coffee with a founder who's, you know, thinking about something, struggling with something, trying to raise their seed round, dealing with their first crisis, whatever. So, you know, like, it's a hard thing to do, to start something, to lead something, to grow something, to deal with a recession, to, you know, like, all the things that you start to confront when you're in charge.
And so, you know, support networks are so important. I think a couple things I would say is a lot of those support networks historically have not been accessible to a great proportion of founders who aren't part of, you know, white males who are, like, predominated in tech. And so, like, these networks have been seemingly quite closed. My wife is a female Arab entrepreneur founder, and she also puts a huge amount of emphasis on kind of supporting women who are starting companies and access to VCs. So, we both just, like, believe deeply in how do we broaden the access for founders or new founders to resources, to introductions, to things like that. So, I sort of, like, acknowledge that, that that's a problem as well. It's a lot, you don't have the network kind of a priori, like, where do you start getting that network? So, hopefully, one thing is just, like, reaching out to other founders. I really believe that. Like, most founders, I want to hope, will take your call just because they, I remember what it was like some of the very first time just kind of reaching out to other founders and cold and just, like, seeking guidance and, like, validation and, you know, anything. And so, I would encourage that. It seems very simple, but do that, and hopefully more of us will take the call.
To the point of, like, when you know. So, I've always seen entrepreneurship for me personally, and others view this very differently. So, I, you know, I can only speak to my experience. My motivation for starting things is really because I feel that we need to build something in response to some societal challenge. And I will have the best impact over the next several years of my life waking up and living and breathing this through this venue as opposed to working on this problem another way. So, system is a very big problem. It's not ours to have. I didn't wake up one morning and say, you know what, the world is highly complex and interconnected, and if only we could understand how everything was related, that would change everything. I mean, Ada Lovelace wrote about this, and Bucky Fuller, and, you know, any number of great scholars, and Stephen Hawking, and on and on and on. Michelle Alexander in writing The New Jim Crow. So, it's, like, across the spectrum, people have recognized the need for taking a systems-based approach. And so, when I was at Spotify, and I sort of knew that this was the next major problem I wanted to work on in my life, I gave my first talk about this at Ars Electronica 12, 13, 14, 15 years ago, and I basically laid out system without knowing that it was system. It took me a long time to figure out how to do this, and, like, is this even tractable? And then it was at Spotify that, like, the tech part started to resonate with me. I sort of felt I could do this. Like, this would be technically tractable, and that a technological solution to this big problem would be high leverage towards advancing systems thinking and systems-based approaches in the world.
I also, at that time, I had spent time at Kennedy School of Government as a fellow working on broadly the place of data, and science, and society. I've spent time advising the UN. I've worked at a big company. And so, I did kind of reflect on, should I just, like, go do this from a university platform, from a research platform, from a social sector platform, from big tech, and then ultimately recognize that, really, the nature of the problem, the nature of what gets built here would be ill-served. It needs to be built, and it would be bad if I built this, you know, at another technology company. It would compromise the sort of integrity of what it is we're trying to do. So, it was really that thing, and then, like, the next week, I was on a flight to Stockholm and talked to Daniel, and that was it. That was the beginning of, sort of, building system.
Hiring for values and culture fit
So, as an entrepreneur, well, I was an entrepreneur, but the thing is I really appreciate that hustle that you have, and one of the questions I had is, you know, starting a company and having one, two, three employees, it's easy to, it's easier to keep that vision that you have for the company and the values of the company, but when you start adding in, say, to 100, 200, 300 people in your company, and as it grows bigger and bigger, how do you, what do you look for when you hire to where you can keep your company's values and missions and visions, but also, you know, with different talent, they have their own set of creativity and beliefs and all that stuff.
Yeah, great question, and that is really something that I spend a lot of my time thinking about. Our director of operations I see is also on this call, and so I spend a lot of time talking with Esther about this as well.
So, it starts with, it starts with how we hire. There are, you know, any number of philosophies of kind of what to prioritize and how much time to take, and especially in competitive job markets and so on and so forth. We've held, and I've been, I've used my founder political capital, let's say, right, to sort of like advocate for this very strongly and ensure this was part of our culture from the beginning, that we do a couple of things that just like extend the amount of time sometimes it takes to really like get to know somebody beyond like the technical screens and stuff like that. But we put a great deal of emphasis on how we discuss our values with a candidate, and once somebody starts a system, they'll get shot, they'll shadow somebody else on the team to go do one of those values interviews, and then they're off to the races leading those values interviews. So, it's really something that we believe it must be something that everybody in the company, especially from different teams, right, like who've never met a candidate, are just meeting this person, and we've just put a lot of time into refining the questions and cases and things that we use to just try to like have a conversation with somebody and get to the heart of, you know, will they be ambassadors for values that we all, never mind just me, we all just believe are core to the culture we feel very proud of.
We also have, you know, introduced kind of a project with our candidates that can be taken any form, but it's a way of us kind of getting to discuss something as a system with a candidate. And those two things, sort of our values discussion and this project around systems for us, and we've talked about so many times, right, like so many times we'll have a conversation with our leadership team about, like, this takes time, and, you know, this candidate is gonna, has like two job offers from two great companies, and I just like really believe that this stuff matters, and like we just have an incredible, incredible, incredible team, and, you know, incredibly low attrition, and like all those things, just like great productivity, great collaboration, and great spirit around the company. So I think it's just like, you know, what are the things that are really important that you just believe are really important, and to make time for those things, and, you know, at the end of the day, competitive markets, all that kind of stuff, you have to be respectful of a candidate's time, most importantly, but, you know, you're trying to build a value-centered company that's going to scale. The only way it scales, the only way I know how I should say that it scales is by putting an over-emphasis on this stuff for the first 10, 20, 30, 40, because then, like, the most exciting, some of the most exciting moments for me over the last, like, year is when I'll meet a candidate during onboarding who I had nothing to do with hiring, and I'm just like, and I email the hiring manager, I'm just like, that was incredible, like the questions they asked me that she asked me during onboarding were just, like, mind-blowingly awesome, and, like, I, you know, and, like, I see two months later, like, she's thriving or something, and, yeah, that just feels great. That just means, like, the primitives are there, and, yeah, you just have to, like, hold on to that stuff if you, if what you're trying to do is build a company that is going to scale and is values-driven.
It's hard for me to, I guess it's hard for me to, like, just share those questions as, you know, because they're not, I mean, it's a long list of things that teams will pick from based on the nature, but I think we generally find that the best way is to try to, like, get to cases, is to get to real-world examples where somebody is struggling with some tension, or having to, you know, make a decision about something, and we're trying to see how they've considered that value in some personal decision, or some life decision, or some business decision, or some managerial decision, we're just trying to get a feel for it, and everybody is coming in with their own, like, very diverse backgrounds, and we have tried to build, like, a very, very diverse and inclusive culture, especially at, like, I mean, at every level of our company, but across the board, it's been critical to me from day one, and so we also acknowledge that people are coming in with very different ways of, and very different, having had different opportunities to manifest values differently, and so we try to just, like, educate our team, and make sure that our team is empathetic to the specific context of a candidate, but also, you know, these are the five things that we really look for, and it's okay if, you know, that's not you, and you'll thrive somewhere else, those are the five that matter to us.
Data science at Spotify
All of it, so we developed an organizational design, and kind of structure at Spotify for that phase, and Spotify as a culture, as a company, really subscribes to even being agile, and kind of responsive in how it organizes itself, so we, you know, would spin up new squads, and deprecate squads, and merge squads, and change things based on where things were growing, and where things were changing, and what we were learning from users and market, and that same philosophy kind of even manifested in how the organization itself, like bigger picture, changes over time, and evolves over time. And so then the challenge was, we were both not, this is the challenge of many organizations, as they kind of go through growth, we were both not able to invest in our kind of core data, analytics, machine learning, platform, culture, and products, while also not able to scale to meet the very nuanced applications of all of that to product, product, and content, and markets, and sales, and marketing, and strategy, and you know, all of the very many functions of Spotify around the world.
And so we built both a large central organization that was working on core platform, core culture, core data tooling, core data sets, core analytics, core machine learning models, all this kind of stuff, and with a very strong emphasis on education again, so how do we help as many people at Spotify as possible get trained in the tools needed to be data literate, so that our work would have the greatest absorptive capacity, ultimately impact, you know, in people's lives and work, and then a whole distributed organization that was embedded in all of those functions, so from, you know, product analytics, like you're saying, and making those A-B test considerations, and helping to design those experiments in a product squad, all the way through to, you know, what's the best marketing campaign to grow subscribers in Japan, all the way through to which new artist is going to break, all the way through to how do we better understand a person's taste in order to create new features that can be the basis for new recommendation products, like Time Capsule, that was an effort to predict music that would evoke feelings of nostalgia for you individually based on an incredible amount of research, an incredible team working on sort of the behavioral sciences of these psychological experiences, and then building algorithms, building models to predict that and help to evoke that for listeners at their choice, so really across the spectrum, yeah, all really, and with a goal of how do we be as thoughtful as possible in how to do this in a manner that was consistent with Spotify's values, but also at the time, I mean, we were sitting and looking at the three biggest technology companies in the world as our competitors, and it was part of the strategy of the company that data would be, and continues to be, one of the ways to materially differentiate and compete, so also with the very real world business challenge of, you know, we need this to go up against Google, Apple, and Amazon.
Handling bad science and evolving evidence
Sure. Thanks, Adam. Really interesting product. I have a question about the occasional revelation of bad science. I think something around Alzheimer's was recently uncovered. Maybe a decade or generation of research was based on bad data. As a consumer of science, I'm wondering if there's a way, or how you would unroll that in your graph, and then how you would surface that to me as the user. Is there a way I can sort of, I don't know, go back in time on certain relationships and see where they diverged, or where they were rolled back, or where you might notify me in the tool that something really big has happened?
Yeah, incredibly important question. One of the things we're most focused on, actually, like active kind of product discussions just in even the last couple weeks for some things we're working on right now.
A few different ways that we're thinking about that. The first is that we believe in presenting all evidence on system as transparently as we possibly can. So, we are showing the work, right? We're showing the source of all relationships that are being constructed on system. We're showing the statistical information that system is retrieving. We're showing any controls that were placed on that experiment. We're showing the strength of that relationship. We're showing all of that. So, it starts with just, right, it's not sort of this opaque, like there is a relationship here, trust us. So, the first thing is just like we believe fundamentally in making sure that all evidence on system is there and available. And our design team spent a lot of time thinking about how to make that as clear and compelling as possible, if you choose to go dig into the underlying evidence.
So, number one is just firm belief. We have a first principle. We have a value. It's codified in like the architecture of the platform that we store all of that metadata. We expose as much of it as possible. And we use design to try to make it as clear and compelling for the user as possible. Then, as we aggregate more and more information about a given relationship, so an Alzheimer's relationship, for example, where we are going now, and you can see this on a relationship page today. So, if you click on any edge in the graph, or if you search for any topic and click on a relationship, you'll see that as there are multiple sources of evidence. And so, we're going to go from tens to hundreds and thousands very, very, very shortly, some exciting things on our end. You will see synthesis coming from system around the extent to which there is consensus or not in the evidence that system has gathered about this particular relationship. And so, when there is actually evidence of disagreement, that will be there, and you'll be able to really see what was the context for that disagreement in the particular paper or data source or particular facet of that relationship.
Then, the sort of final piece is that we are very interested in the evolution of these relationships over time. So, every piece of evidence on system has its timestamp of when it was gathered in the world as well as when it was added to the system, of course. And so, in the future, you will be able to see the evolution of information on the platform, the evolution of consensus, the evolution of value in a given relationship. So, we're working on some interesting things there that I'm quite excited about. The final piece is that we built a community around the platform of system thinkers who come from all walks of life, from healthcare to education to government to climate change to technology. And we have our Slack community, and you're all very much welcome to please join us. And if you feel passionate about the applications of systems thinking in the world, please join the conversation. And so, we will be introducing tools to then bring the power of community into these areas like biomedical research or climate change or elsewhere and apply that extra level of kind of human filter on top of the technology that we've built. So, we're learning a lot from platforms like Wikipedia on how they deal with these kinds of things. And these are not error-proof platforms by any means. Great scientific journals have been duped in these kinds of things. So, the best thing we could be doing is these are the steps we're taking.
Advice for transitioning into data science
Yeah, great question. Yeah, I mean, my general belief is that it's very helpful to start, especially given kind of my own interest in data science and what it can be used for most meaningfully in a company, in society, in government, in a hospital, is to really start with statistics and make sure that you invest the time before you're kind of working with data, before you're building things, to really just, yeah, get indoctrinated with sort of core principles and statistics. It will serve you beyond anything that, you know, any new tool or technology that you will adopt, in my view, because it will give you the shared critical lens on information that will carry you throughout your career, whether that's in the field of data management or marketing, you will have taken the foundational, like data science is a science at the end of the day, and we have spent a huge amount of time for very good reason over the last few years as a field, focusing on the engineering side of data science and like all this great tooling, because we needed to like just deal with all this data and figure out how to build models and deploy them into production and then deal with, you know, whatever, all the stuff. And now we've got a plethora of things to go use, but the science is the thing, like what's the real value added? If you think about it in three years from now, when you're sort of about, you're a CEO, and you're evaluating, like what's the level of investment we should be making in data science versus whatever, it's going to be based on the quality of those insights and the actionability of those insights and the value that comes from the products and models that are being built. But for those first couple, I mean, like at the end of the day, it's just making sure that there are core scientific practices and good scientific rigor. You are thinking rigorously about what you're controlling for. You're thinking rigorously about bias in the sample, and through that, ensuring that you don't introduce further bias into the decision making in the company. You're thinking, hopefully, as we are advocating for taking a more systemic approach to looking at a problem and thinking about it more holistically, what is impacting it? What is the path to causal inference in this particular problem area? So that's where I would start and really like hold on to those things. And of course, then the rest of it is a little bit more, you know, there's great resources and great things to then educate yourself on. But yeah, stats, I would just like really emphasize that.
And now we've got a plethora of things to go use, but the science is the thing, like what's the real value added?
Yeah, I don't feel like I have much to add to that besides, like, that's the first step, right? Like, before you jump into how do I do this? Or how do I learn to code? Or how do I learn statistics and machine learning? Like, ask yourself why you're doing it and how you learn best, because everybody is completely different. And you're going to get 5,000 different pieces of advice on how you should do it based on how someone else did it. And they're not you, right? So, do you learn best in a classroom with classmates and a teacher and somebody that you have access to to ask a lot of questions? That's me. Maybe you need to take a class, maybe an in-person class. Maybe that's not you at all, you're completely autodidactic. Then read a book. You're going to be happier that way. Just stay true to what works for you and don't get sidetracked by what everybody else tells you you should do.
Yeah, sure. Thanks. Yeah. So, basically, on the podcast, I just talk about my kind of journey into data science. And I actually come from a psychology background. So, anyone who maybe comes from more of a qualitative background trying to get into data science, maybe that podcast would be helpful.
I would also just add that, especially for those who come from either qualitative backgrounds or from sort of natural sciences, it's incredibly valuable, in my experience, to hold on to those skills that you nurtured in those areas because data science is a fundamentally interdisciplinary field in its best form. And so, if you orient visually, then you will be a great, you know, advocate for the merging of data science and data visualization. If you orient qualitatively, you'll be a great advocate on your team for the synthesis of user research and product analytics. If you orient towards natural sciences, you'll be a great advocate for thinking about causal inference and is this a real systems view of the problem and is the experimental design appropriate for what we're trying to do with it. So, know that, really, you can come at it from really any angle and what you come with it, what you come to it with, will be very valuable because it's a big tent and we'll benefit from all of those prior experiences.
System's five-year vision
I hope in five years from now, system is a resource for decision making in every field and around the world where you come to system either through our platform itself or through APIs that we will expose to points of decision making as we grow. So, wherever it is you're trying to make a decision about your health in the context of meeting with a doctor or doctors interacting with each other, in the context of a business where you're trying to figure out what's changing and moving a metric, in the context of the economy and we're trying to figure out what policy decisions will move a particular social factor, in the context of global health where we're trying to understand and anticipate what could be giving rise to what changes in a particular metric might impact an emerging epidemic in a country. The thing that unifies all of these things right now is that we are so siloed in the way we see these problems and we simply don't have the tool, we don't have the ability to go access the context for the decision we're trying to make. We're not able to go query for what are all of the things we already know are related to our driving or being driven by that thing that is so important to you, to the city, to the country, to your team, to the world at that moment. And so, our vision is really to empower everyone to be able to access that system's view just as easily as you can go on Wikipedia or go on Google today and get information about it. We want to make it even more accessible for you to go understand the system around it, the context for it.
Just have a quick question, if that's okay? Yeah, sure. Adam, do you use RStudio in your company? We do.
There's a lot of Python as well and, I mean, generally we're working across a very large stack and so certainly in the data science realm we're working in R and Python, but we're doing natural language processing, we're doing information retrieval, we're building entity resolution systems, we're doing data visualization on the front end and doing a lot of work there, obviously. We're building search, so there's a lot of parts of the stack and obviously lots of languages across the board and, yeah, we try to be promoters of as much openness as possible and open source as possible. We ourselves will be making much of systems for our charter, all of the data on system, all the metadata will be fully open and accessible, so we also want to embrace and support advocates for openness in tech.
Thank you so much, Adam, for joining us today and sharing your experience and the great discussion. I do just want to ask if people want to connect with you, is LinkedIn the best way or what's best? Yeah, LinkedIn's great, that'd be the easiest thing. Well, thank you so much. Thank you, everybody, for joining today, too. Great to see you and we'll be back same time, same place next week.
