New data science tools & old laptops on fire | Jenny Bryan | Data Science Hangout

Transcript#

This transcript was generated automatically and may contain errors.

Hey there, welcome to the Posit Data Science Hangout. I'm Libby Heron, and this is a recording of our weekly community call that happens every Thursday at 12 p.m. U.S. Eastern Time. If you are not joining us live, you miss out on the amazing chat that's going on. So find the link in the description where you can add our call to your calendar and come hang out with the most supportive, friendly, and funny data community you'll ever experience.

I am so excited to introduce our guest today, Jenny Bryan , Senior Software Engineer here at Posit. Jenny, thank you so much for being with us here today. I would love it if you could introduce yourself, tell us a little bit about what you do, and something you like to do for fun outside of work.

Thank you for inviting me, talking me into it, and for all of you folks to show up. I feel like I've been a little bit hermit-y ever since COVID, and so this is a bit of a starting to do more things like this again for me. So I have been at Posit, or previously RStudio, since 2017, working for the most of those years as someone who helps maintain packages in the tidyverse , among other things. And I co-authored the second edition of R Packages with Hadley, and I give talks and workshops. So those are sort of the things I do. But for the past, I guess, about two years, I've been part of a small chunk of the tidyverse team that's been lent out substantially to the Positron team, so the folks developing this new IDE. So still working some on R Packages, but frankly, a lot less than previously and spending more time on things Positron-related.

And I think another relevant thing, if we end up going in career directions, is that before this, I was a professor at the University of British Columbia in statistics. And then before that, I worked in management consulting. So I feel like I'm kind of on a third career, and I have definitely taken the scenic route to what I do now, which is, you know, I think overall very good. But then it also gives you some like, oh, I'd be better at this now if I had done it, and only this for 30 years or so.

What about something you like to do for fun outside of data work?

I knit. I guess that's one of my, that's a big hobby of mine that's come and gone over the years. But I think I live in Canada, but I am originally from the US. And somehow my reaction to the past several years in the US has been to knit more to get my mind off of things and do like soothing, repetitive tasks that have a tangible output. And like, that's like linear effort in, results out. Yeah, you need that. That's also a good counterbalance to coding, I think. Like that you can, with knitting, you can be fairly sure that the amount of time you put in is reflected in some sort of physical object, which is really nice.

Tools data scientists want most

Jenny, as a builder of the tools that data scientists use, I'm wondering if you have any insight into how you see the demand kind of shaping up in terms of the tools that data scientists now want to use most. For example, when RStudio came out, like the being able to see your environment while you were coding was a huge thing. And then the viewer was another big thing. So what are some of the things that you see really kind of being asked for a lot or kind of building a lot of demand for?

LLMs. Everyone wants to be able to get the LLM of choice in their brain and like focused on their problem and contributing ideas or code. That's what it feels like to me. Like the thing, the main thing that comes to mind is that that, you know, is just like totally table stakes for any tool. For any tool that if there is any LLM relevance, people want it and they want it like yesterday.

And then I guess because I mostly think about Positron these days, you know, just like helping people create and maintain like very healthy like environments. You know, having different, the different tools that they work together, play nicely together, it's easy to like stand something up and getting it, the whole system working well together. But I feel like especially now that I'm working on an IDE that serves Positron as, sorry, Python as much as R, I'm much more exposed to like the problems of Python people, which definitely includes standing up environments constantly. And so then I'm also like thinking about that more for R users as well, just because it like simply comes up more in a polyglot user base.

Adapting to change and LLMs

I would like to ask a question because it's something that I've been thinking about a lot with Positron and it is just the adapting to change part. Like has, have you experienced needing to adapt to change recently and how have you embraced it? Because especially with LLMs being a topic, I am an LLM curmudgeon and I'm having a hard time adapting. Give us some wisdom there.

Well, so sometimes crying helps. I like definitely when I first started working on Positron and I'm not just trying to like be funny. Like I had multiple episodes of just like sobbing and frustration. And, but I was comforted by the fact that someone named Brian Lambert , who's also on the team, who was there much ahead of me and works more on like user interface front end stuff and not, not so much on our stuff. But he shared that he too had cried multiple times about the project because it's a very challenging code base to get to know, even if you are already fluent and all of the technologies involved, which I most definitely was not and arguably still am not. So like it's just plain hard and whatever you need for emotional catharsis and that might be sobbing occasionally helps.

But it is true that like the LLM, the relevance of LLMs has changed dramatically over the course of those two years, like really dramatically. And so I think I, I both got more competent and I got more comfortable with being uncomfortable, but also the ability now to chat with CoPilot or let Cloud Code fly on something has helped immensely at like feeling like I can make a little more progress quickly or like see something that works. And even though it might not be the way I want to do it, it like proved to me it's possible and like, here's, it gives me something to shoot at, a critique or whatever.

Early on in the LLM hype cycle, I mean, we're still like massively in the hype cycle. I was super skeptical because I was like, I can write R code much better than this thing can, like much better. But my attitude changed a lot when I applied it to problems where I need to write Rust or TypeScript, which I'm substantially less good at. And certainly in the case of TypeScript, LLMs are really good at. And so I think part of one's skepticism about using these tools can also come from like, well, how much are you expanding your comfort zone? It's like, if you stay in your comfort zone, the relative gain might be small, depending on what your comfort zone is. And so I think one way to use them, or at least to understand why other people are using them, is that they are getting outside of their comfort zone more often. And then having this tool that helps explain code to you or helps generate code to you and like, you know, there are better and worse ways of using it, is a real accelerant. And so sometimes I think skepticism comes from a place of, you're not using it on the same types of problems or in the same way that the enthusiasts are.

Sometimes I think skepticism comes from a place of, you're not using it on the same types of problems or in the same way that the enthusiasts are.

Laptops on fire and career changes

I see some posts on Blue Sky about Jenny setting laptops on fire. What is that about? And then a serious question. How did you decide to change your career a few times?

I did not. I did not. And I still think overall it, it has been a good thing. It's not universally good, but what happened is I gave a talk in New Zealand. And it had a lot of, like, workflow and process stuff in there. And I had two slides that were very provocative. Like, one was, you know, if I open up your project and I see, like, set working directory, blah, blah, blah, I will set your, come to your office and set your laptop on fire. And the other one is if I, the top of your script is rm list equals ls. And Hadley took photos of those two slides and tweeted them. And this was, you know, back in the heyday of rstats Twitter. And yeah, people lost their ever loving mind. Like, it was very, very upsetting or yeah, reactions all over the place, but definitely a reaction.

And so I then felt the need to write a blog post about, like, clarifying what I was talking about. And so that's this blog post called, like, Project Oriented Workflow, I think. And every time we look at the analytics for the tidyverse blog, that thing is doing numbers to this day. Like, it's incredible. So I had no idea when I wrote that, that it would attract as many eyeballs. But I think part of the reason why the people had such a strong reaction, both, you know, negative and positive, is it can really feel like a real attack, a personal attack on one's code. And so that was the part that I think is regrettable. But it got people's attention.

And so part of I think what, you know, is upsetting or about that type of provocative way of stating things is it feels like someone's taking away a piece of my workflow that's actually super important. And they haven't told me what I could do instead. So that's really what the blog post was about. Because I, too, used to do both of these things, because I felt like it was, like, the only way to, like, pivot my attention from one project to another or, like, build paths that worked for me in the next five minutes. And so I think the important thing about what I was able to explain in the blog post is I do want to take those habits away from you. I stand by that. But it is important to, like, acknowledge they're solving a very real problem. I just think there's a better solution to those problems.

And, you know, the one about not clearing your workspace all the time is, like, every time you clear the workspace, it's a pretty good sign that you should at least just be restarting R. Or you should be, like, switching your attention to a different instance of RStudio or to a different window of Positron. It's that those are better ways of sort of pivoting your attention from one project to another or sort of power cycling an existing project. And then the issue about paths is that you want your projects to travel gracefully across space and time. And so you want to build project paths relative to the project. And we have better tools for that as well. And I think the here package is a great implementation of that. You don't need to use it, but it can help tremendously.

So I was a statistics professor, but, like, even in grad school, it was very apparent to me that I really enjoyed any sort of software development and was, like, less excited about proving theorems and did, like, the bare minimum of the math part and, like, way more than the minimum of the programming part. And you can navigate through statistics academia that way, but you'll never be regarded as, like, truly awesome unless you're Hadley, but then he left academia, too. And then that theme just, like, carried on throughout my entire statistics career. I did manage to get tenure. I was an associate professor by continuing to do, like, a ton of data analysis, actually is what I mostly did, some working on tools for data analysis, but really more just doing data analysis.

So once I got tenure, I decided to the way I was going to implement the freedom that that gave me was to totally focus on software because I was, like, I don't feel bad about this. I feel like it's professionally, like, ethically okay. It means no one will regard me as just killing it in my job, but they will certainly tolerate me. And then I, like, either that's just the way it'll be, like, maybe I'll never be promoted to full, but I'll be happy doing my software development work, or maybe it will give me the chance to go do that for real somewhere. And then I was lucky that the latter came true. And so when Hadley first started building, got to build a tidyverse team, I was able to come join that along with, you know, a couple other people around the same time.

And then that felt great. It feels so great when doing what you really want to do and where you think your comparative advantage lies actually aligns with what your, like, job description is. That's been amazing. You don't have to have that in your life because I was willing to just play my cards the way I wanted to in academia and not worry about the misalignment, and which sometimes that's just what you need to do, but it is really awesome when you can get those things aligned.

LLMs as a tool: using them well

How do you navigate the law of the instrument when working with emerging tools like LLMs? I've noticed parallels with Shiny where it sometimes gets applied to problems that it might not be best suited for. So how do you balance enthusiasm for new tools with thoughtful tool selection?

I think LLMs are like a chainsaw. Like, it really is very easy to like, open an artery and just like, to use it either on something that you shouldn't, or I'm less worried about that, but to like, trust it too much. So I will just say that I try to be just super bossy with LLMs, and very critical of their output. So I've only done something that I would say was vibe-coded once. So I much more ask very targeted questions, and I'm pretty willing to be like, are you sure? Or like, this doesn't make sense to me. Or just start the conversation over if we've gotten to some local minimum, like, of quality. So I don't know. That's my main answer is they are super helpful, but yeah, sometimes it's also true that you need to just back out and think about the problem yourself.

But I think if you use LLMs in that way, like you really think about your prompt, you really think about the context, and you really critique each answer, they end up acting like a giant rubber duck. And so, and that can be very healthy, because it's making you articulate, this is my problem. These are the constraints. I want it to look consistent with this code base. I'm worried about building a very inefficient solution or something like that. But like forcing you to say those things actually forces you to think those things. So I think certain types of habits in your LLM use can actually force you to refine your own thinking, whereas before you might have just started coding, right? So I think there are ways to use them that keep your brain like plugged in.

If you use LLMs in that way, like you really think about your prompt, you really think about the context, and you really critique each answer, they end up acting like a giant rubber duck.

Positron vs. VS Code for data science

Why should I learn Positron instead of VS Code?

Oh, because if you like having an interactive REPL right there, for example, like so the R console that you can send code to, I guess this is possible in VS Code, but involves like a lot more fragile setup on your part. So having that be a just basic core feature of the IDE for a practicing data scientist who uses R or Python or R and Python, I think that's huge. Like the way the console is feature one in Positron, then I think also after that comes things like the environments pane, the plots pane, like all the sort of data science specific tooling that's just the core product as opposed to things you might need to sort of configure and get working together on one day and then sort of actively maintain to keep them all playing nicely together. I think those are the main reasons for Positron over VS Code.

We should add. Did you mention the data explorer? I was just going to say data explorer. Oh, okay. Yeah. So having, you know, once you've got a data frame in one of your R or Python sessions and you have a way to like fling it into this data explorer through different gestures and look at it like it's a spreadsheet, um, which can be super handy. You know, the way I use it is when what few data science tasks I still do is frequently I need that kind of tangibility to write my code or to figure out why my data wrangling code isn't working and can use that sort of spreadsheet to like stare at the data or like get a few rows at the top that I feel like are causing me problems to better create my code.

Yeah. And I feel like each of those elements is like sort of possible in VS Code, but it's like a lot more janky and it's a lot more on you to get individual extensions or things working together. But yeah, when I see these questions, when Positron is discussed in places I'm like, Jess, why didn't you just use VS Code? I'm like, are you serious? That's like my actual response to that because I'm like, it is so different. So it usually tells me like this person doesn't do a ton of data science or they're like completely notebook focused and they're getting all that stuff sort of from notebooks and they don't understand, they don't know what they don't know, is my reaction when I see comments like that. I'm like, that tells me that you've never worked in this way and experienced how awesome it is.

Technical writing and Happy Git

Jenny, I would just ask a question. Happy Git is like one of the things that saved me the most when I was in grad school. Like I read it religiously. I gave it to everyone. I printed it out. And I'm always like amazed how much I come back to it, like even like five years from now. Can you tell me like how you write like technical documents? Like how do you like view the audience? Like do you, how do you gauge the skill levels? And do you have any tips on how to like, you know, talk about these kinds of really technical questions or technical skills to like non-technical audiences?

Oh, one like principle that like definitely comes to mind for me is if there's something I needed to know to work, it needs to be in the talk or the book or the post or the docs. And if there's something I have never needed to know, it does not need to be there. And that's like super clarifying because I think often when you write docs or books or articles or whatever, it's so easy to be like, you pull way back and you're like, let's start at the very beginning. And you go off and like do research on things that like you've never actually needed to know. And like, if you're writing, you know, the definitive account of something, you do need to do that. But most people don't want to read the definitive account of things. And so I kind of feel like if there's something that you've never needed to know, let's say about Git, since that's what we're talking about, and you operate like at a pretty high level in Git for 10 years, it's pretty safe that you don't need to mention it now, or at least you have the permission to not mention it now.

And likewise, there's often gross little moves that you need to make that actually help in your daily life, but they feel like too small a detail to mention, or it's embarrassing that you do this. And then I'm like, no, if it's like some move that you need to make on a regular basis, even if it feels inelegant, you should probably write about it, because it's friction that actual people do encounter. So letting your own experience of a tool, giving yourself permission to like actually write about it that way.

So a lot of what's in HappyGit, like the way the website came about is back in my prof days, we would drag classes of 100 or more people every semester through the fun experience of getting Git and GitHub set up. And so we ended up with this just like battle-tested set of instructions and a very good list of like problems that people have. And then we turned that just sort of collective, like super painfully gained wisdom into that resource.

And then I think another phrase that's really helpful comes from Greg Wilson of Software Carpentry, which is never let truth get in the way of clarity. Like letting yourself write things sometimes that are not like absolutely, totally, completely pedantically true, but it's like, it's the truth that actually matters. That can also be very freeing to write things that are like, reasonably short, readable, and will work for most people most of the time, but, you know, is glossing over some technicality or something like that.

Never let truth get in the way of clarity. Like letting yourself write things sometimes that are not like absolutely, totally, completely pedantically true, but it's like, it's the truth that actually matters.

New data science tools & old laptops on fire | Jenny Bryan | Data Science Hangout

Transcript#

Tools data scientists want most

Adapting to change and LLMs

Laptops on fire and career changes

LLMs as a tool: using them well

Positron vs. VS Code for data science

Technical writing and Happy Git

R package development in Positron

Getting started with R package development

Individual contributor vs. manager career paths

Featured software#

air

positron

tidyverse

tidyverse.org