AI Coding Assistants: Hype, Help, or Hindrance? (Rebecca Barter, Arine) | posit::conf(2025)
AI Coding Assistants: Hype, Help, or Hindrance? Speaker(s): Rebecca Barter Abstract: AI coding assistants like ChatGPT, GitHub Copilot, and Codeium promise to revolutionize our coding workflows—but how useful are they in practice? Are they our new overlords here to take our jobs? Or just a passing gimmick? I think the reality lies somewhere in between, and that understanding these tools is key to staying relevant in today’s rapidly evolving data science ecosystem. In this talk, I'll show how I’ve used these AI tools in RStudio, Positron, and VS Code to speed up both my advanced R workflows as well as my learning experience as an intermediate Python programmer, providing examples, pitfalls, and best practices. Materials - https://rlbarter.github.io/posit-conf-2025/#0 posit::conf(2025) Subscribe to posit::conf updates: https://posit.co/about/subscription-management/
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hi, everyone. Can you all hear me okay? Awesome. My name is Rebecca. I am a senior data scientist at Arine, as well as a professor at the University of Utah. And I am super excited to be here today to talk to you about AI coding assistants, hype, help, or hindrance.
Okay, so probably most of you are aware that a few years ago, something happened. OpenAI released ChatGPT, and everything kind of changed. So probably even if you haven't used an actual coding assistant, at some point you've gone to your preferred kind of ChatGPT analog and asked it to write some code for you. So here I am asking ChatGPT to give me some, you know, QR code that I can then copy and paste into my IDE. You know, it might not have all the context, so I probably still have to make some modifications to make it match my actual data variables and stuff like that. But it's pretty cool. It's pretty sophisticated, right?
Like I already am impressed by this functionality, right? And I'm sure many of you have been as well. So what some of you, but most of you probably were aware, some of you may not be aware, is shortly after ChatGPT was released, GitHub released GitHub Copilot. So they're kind of seeing ChatGPT, and they're kind of like, well, this is awesome, this is a language model, it can generate code, but wouldn't it be awesome if we developed some kind of LLM that's specifically designed for code and is embedded inside of our IDE?
So in this talk, I'm going to be showing you a lot of examples from Positron Assistant, because we are at PositConf. But under the hood, Positron Assistant is actually just GitHub Copilot and Anthropic Cloud. So everything that I am going to talk about while I'm in the Positron IDE using Positron Assistant is going to be equally applicable to if you're using either of these two tools, Copilot or Cloud, in, say, VS Code or PyCharm or whatever IDE you use, except for RStudio. RStudio, unfortunately, has quite limited AI capabilities. It has code completion, but that's kind of where it stops.
Hype: the agent demo
So is our, like, these AI coding assistants hype, help, or hindrance? Let's start with hype. So here I am in Positron. On the right-hand side, I have just, like, a simple Quarto document open, and on the left-hand side, I have my kind of Positron Assistant window. So what I'm going to show you is a thoroughly irresponsible example of how I do not recommend you use these tools, but it should give you a kind of a sense of what these tools are capable of right now.
So if you look at the bottom of my Positron Assistant window, if you're at the back of the room, maybe good luck, but you can see that there's this kind of prompt box, right? And at the top of that prompt box, you can see there's several context documents. It says add context, so I could upload documents if I wanted. It's got my console session, so it can see everything I've run in my console, and it's got this Quarto document that I have open called cost reports analysis, where so far, all I've done is loaded in this kind of data set that I'm calling hospital costs.
So I'm essentially going to ask Positron Assistant, in this case, I'm using the agent, I'll talk more about that in a minute, but I'm going to ask it to do my job for me. So I'm going to say conduct an analysis of hospital costs in this Quarto document, and I'm going to give it an additional prompt that says be sure to check the names of the variables before you begin, because if I don't do that, what do you think it does? It hallucinates a bunch of variable names, and we get nowhere.
So I'm going to send this prompt, and I'm going to see that the first thing it does, it thinks for a little while, and it says sure, no problem, I can do that, it always says that, even if it can't do it. And the first thing it says, you can see it says using inspect variables. So this is where the agent is kind of coming into play. The agent has agency. It can do things. It can look at your data set. Right? That's cool. It can look at your data set and be like, okay, what variable names does it have? So it can use that information in it's kind of like the output that it gives you.
So, okay, inspected my variables, then it kind of tells me some other things, like, oh, here are your variables, I can see you already investigated inpatient revenue and Medicaid charges, so it's actually I did that in the console, it's going to focus its exploration on those, using that context from what I did in the console. And it kind of tells me some other things, and eventually, after it kind of thinks for a while, it creates this entire new Quarto document that has all of this analysis in it.
So rather than scrolling through this Quarto document that it's created on the right, I'm just going to flip to the top, I'm going to go and render this Quarto document, and let's just scroll through and see what it looks like. So this AI has just done my job for me. Awesome. Like, it's doing a missing value exploration, it's kind of summarizing various things at the state level of all of this hospital data, it's creating kind of a nice visualization of, like, well, how many hospitals are there in each state? You know, it's created this really sophisticated document, and all I did was, say, conduct an exploration.
So I'm going to pause here, because if there are people in this room who have never played with these tools yet, you might be feeling a little uneasy. I know when I first started using these tools, I kind of felt like this, like I had a lot of existential dread. I was like, oh, I've wasted my career, I spent all these years trying to learn how to code, and now AI can just do it for me.
So at this point, I've been using these tools for quite a few years, and that existential dread has definitely faded. I think, like, rather than this AI taking my job, it's really just changed the way I do my job. Right? Instead of it doing my job for me, I just kind of use it as a little assistant to do the things that I don't want to do.
rather than this AI taking my job, it's really just changed the way I do my job. Right? Instead of it doing my job for me, I just kind of use it as a little assistant to do the things that I don't want to do.
So while we live in this world where all of our vibe coding friends, who have zero coding skills, can, like, make an entire application, like, using AI on their weird AI backwards laptop that Gemini made, it kept doing this, even though I was like, flip it around, dude. I'm like, we're rolling with it. Even though they can do this, like, and it looks really impressive, like, within a few days, they kind of end up here. Right? They end up in this situation where they run into a bug or an error, and they're like, everything is broken, and I don't know how to fix it, and everything burns to the ground.
So are these AI coding assistants meeting the hype? Well, kind of. They're really cool, but they're probably not going to take our jobs yet.
Help: the full toolkit
However, let's move on to help. They are so helpful. So to show you how they're helpful, all I've shown you so far is the agent and, like, one kind of big, vague, irresponsible example of the agent. So I'm going to take a step back, and I'm going to show you what else these AI coding assistants can do. So they can do code completion, inline editor, they have a chat, agent, and a file editor. So I'm going to kind of go through and give you some examples, especially for those of you who haven't seen these. And it's kind of going in order of increasing sophistication, so I actually use these simplest tools probably the most.
So let's start with code completion. So here I am in a Quarto document. I've written some code already, and at the bottom of my second chunk, I've got a code comment that just says create a scatter plot of two variables. Honestly, I could not even write the variable names because I've already used them above, and the AI is going to be like, I know what variables you want to use. So all I'm going to do is I'm going to hit enter to start a new line. So here I go. I've started a new line, and all of this ggplot code has appeared. I did not write this. This is a suggestion or a code completion from the AI, and I can choose either to ignore it by just continuing to type, and it will kind of keep trying to suggest stuff as it figures out what I want to do, or I can accept it by hitting tab.
And if I hit tab, now this code is part of my document, as if I had written it myself, which I didn't, because I'm lazy. But because it's part of my document, and I'm in a Quarto document, I can then run this code. So I can see what the output looks like. So I can run all my code, and you can see it's made this fairly nice plot. I could do various things to it if I wanted to, but the thing that I see about this plot is, like, oh, my God, it has scientific notation. I hate when R gives me scientific notation, and I can never remember how to get rid of it, ever. Like, for the life of me. I've probably done it so many times, I have to Google it every time. But no longer. Google, see you later, because now I'm going to use the inline editor to get rid of the scientific notation for me.
So I'm going to highlight my code, I'm going to use a keyboard shortcut to pull up this little prompt bar, and I'm going to say, get rid of the scientific notation, and scale the axes by a million. So I'm going to send this little prompt to the AI assistant, and let's see what it does. So it's going to think for a minute, it's going to prepare some edits, and you can see it's kind of modifying my code, and it's highlighting whatever it's changed in green. If you can see, there's some slightly darker green sections, that's what it's adding. So it's dividing my variables by a million, and it's adding these, like, scale X and Y layers, which is pretty cool, and it's using this, like, scales comma format function. That's what I can never remember. It's comma format from scales. But it's okay, I don't need to remember it. AI can do it for me.
So I can choose to either reject these changes or accept them. If I accept them, then I can go up here. I accept them, and now this new code is just a part of my document, as if I had written it, but I didn't, because I'm lazy. So I can now rerun this code to make sure it works, and here we go. Now my scatterplot has no more scientific notation. It's added millions into the axes, and I didn't have to do a thing, except tell the AI what to do. Pretty cool.
So let's move on and talk about the chat, the agent, and the file editor. So I've bundled these together, not because they are the same, but because they all live in the same place. So the chat, I'm not really going to show you. The chat is essentially like, you know, GPT, but it's context aware. You can ask it questions, but it can see your file and your console, and you can add extra context, and it's actually really, really, really useful. The agent, I already showed you, because I asked it to do my job for me. Probably shouldn't have done that, but, you know, the agent's pretty cool. The agent has agency. The agent can create entire new files. It can do its own explorations, the results of which it can then use in whatever it's kind of outputting to you.
And so I'm going to talk about the third thing here, which is the file editor. So to get to the kind of assistant panel that houses the chat, the agent, and the file editor, in Positron Assistant, you go over to this kind of little AI icon. In other IDEs, it'll be a different icon, but it's pretty much the same. So let's go over here, open up that kind of AI assistant window, and you can see down here, there's a little menu where I can choose, do I want, you know, the editor, the chat, or the agent? I'm going to stick with the editor, and I'm going to write this prompt that says convert all of the code in this Quarto document to base R instead of the tidyverse. Why would I want to do that? Because I have a colleague who hates the tidyverse. Like for real, I did have a colleague that hates the tidyverse. And so really, I can just ask AI to do this for me, because for the life of me, I cannot remember how to do anything in base R at this point.
So let's see what the AI does. So I've got all my kind of tidyverse code. When I send this prompt, it's kind of thinking for a while. It kind of outputs this diff. So if you've used GitHub before, this should look kind of familiar. Everything it's highlighting in red is what it's going to delete. Everything that it's highlighting in green is what it's going to add. So you can see it's deleting all of this tidyverse stuff, and it's replacing it with all of this inferior base R stuff.
So I can again, I can choose to either reject all of those changes or accept them. And if I accept them, now my document just has all of this kind of base R code. So that was pretty cool.
What these tools are actually good for
So clearly, these are pretty, pretty useful, right? All of these tools. So I kind of want to give you some other examples of things that I have used these kind of AI assistant features for that have helped me a lot. So code completion, I find it incredibly useful for remembering functions whose names I can't remember, which happens a lot, and for discovering new functions that I didn't know about. I particularly find this useful when I am at an intermediate coding level, especially like, you know, if I'm using Python, for example, where it's like, I can't always remember all of the syntax or the names of the functions or the methods that I want to use.
If I'm at more of an advanced level, sometimes I find this kind of annoying, because I know exactly what I want to do. So it's, you know, hit and miss. Code completion is so useful for generating repetitive code. You know, when you have to write some code, and then you copy it, and you have to modify one thing in each line, and you're like, did I get them all? Did I make a mistake? This just is so good at detecting patterns, and it does it for you, and now I'm like, I know I didn't make a mistake, unless the AI made a mistake, which it might have, because it's not perfect.
The inline editor, other than just using it for just, like, generally editing my code, I often use it to, like, scale up from a test example. So let's say I wrote some code to, like, do some specific action, but I really want to have, like, a generalized function that I can then apply to, like, either other data sets or other columns or whatever. I find it really useful for that kind of thing. Like, I write the example, I say, turn this into a reusable function, it does a great job.
The file editor, which is what I just showed you, I use this a lot to, like, optimize my code, to refactor my code. So I wrote a book, and I wrote all of the kind of code for that book using R, but I also wanted Python version. So I was like, here are all of my R files, just take these and, like, convert them to Python with pandas, and it did such a good job. It was, like, saved me, like, literally months of work.
The chat, I use a lot for debugging. So debugging is, like, you know, I've got this, like, error message, and I don't understand why I'm getting this error message. So I ask, why am I getting this error message, and it can see my code, and it's like, oh, it's because you didn't use AES in your ggplot or whatever. I also often ask the chat to explain code that I don't understand. What is this document doing? And it will just give me a summary. It's really, really helpful.
The agent, which is that kind of fancy example I showed you at the very beginning, where it can make entire new documents and kind of do its own explorations, I sometimes use this to, like, generate unit tests, to create documentation. If I have a complicated project, I'll just be, like, create a readme document for this entire project, and it will do that. I often use this also to create project templates. So, like, a few weeks ago, I literally, like, I needed to make, like, a Python for Shiny for Python little, like, kind of dashboard, and I'd never used Shiny for Python. And so I was, like, okay, agent, make me a template based on what I want to do, and then I used all of these other tools to, like, fill in, like, all of the stuff, because I don't know the syntax, right? This whole thing took me three hours, and it would probably have taken me, like, at least a week if I didn't have these AI tools.
So are these AI coding assistants help? Definitely. I would say I'm probably at least three times more efficient with AI, and it depends on what I'm doing, though. Some tasks, that's an over-exaggeration. Some tasks, that's a dramatic under-exaggeration.
Hindrance: the downsides
Okay. Lastly, I want to talk about hindrance. So code completion can be kind of annoying when you really know what you want to do. So if you are an advanced coder, you probably find code completion super annoying and you've turned it off. So when I am using a language I am not as good at, I find it really helpful. When I'm using a language I am super good at, I find it a little bit less helpful.
AI introduces bugs all the time. And this is another one where your coding level kind of comes in, because the better you are at the programming language, the less of an issue this is, because you can see pretty quickly what the issue is and fix it. But if you don't know the programming language, everything is going to burn to the ground pretty quickly. It also creates inefficient code. Remember that this is just an LLM that is, like, learned from code that's on the internet, and code that's on the internet is not always good. So the code that it gives you is not always the best possible code you could be writing.
Another, this is one that I struggle with a little bit, is over-reliance. So once you start using it, it's kind of hard to go back. It's like when your teacher was like, oh, you're not going to have a calculator in your pocket, you need to learn how to do this yourself. And it's like, yeah, but do you? Like, these tools aren't going anywhere, right? So I have, like, mixed feelings about this one.
And then I have to mention the security vulnerabilities. So if you work with sensitive data, you need to be aware that anything that your AI can see, you're essentially giving to the company that owns the AI, right? And that's a problem if you have sensitive data. There are licenses that kind of, you know, protect you from that, and, you know, so you just need to be aware that, like, be careful with what you share, what you give your AI access to.
So are these tools hype, help, or hindrance? I think they're kind of all three, but, like, mostly help. And I would love to take any questions. Here is just a QR code that takes you to these links if you want to connect a little further later.
Q&A
Okay, there's a question that says any best practices for documenting data to support the AI's understanding of it? I think this is a question that is more a question of, like, documenting in general. Like, I think having thorough code books and, like, data dictionaries is just important in general. So I would say, like, I've definitely done it where I've given it, like, I've uploaded as context, like, a data dictionary. And the better your data dictionary is, the better a job is going to be able to figure out what you actually need. So I don't think that's necessarily an AI specific thing. I think it's just document well in general. And sometimes that's not a thing AI can do. You need to do that, right?
