commons

AI Agents for Data Analysis

posit-dev/commons

posit-dev.github.io/commons

24 stars

1 forks

R Python

Artificial Intelligence

Data teams typically have trusted code that they use to analyze their data and build reports and apps. commons leverages these trusted calculations to create accurate, fast, and cost-effective agents. Trusted calculations can come from R and Python code, data dictionary definitions, Snowflake semantic views, or Databricks metric views.

When answering questions, commons agents first search through a pool of trusted code. If the agent finds an appropriate piece of trusted code, it invokes that code directly. If it does not find relevant trusted code, it searches through additional context before writing custom SQL, R, or Python code. Answers display provenance markers deterministically assigned based on the analysis path.

Contributors#

Resources featuring commons#

Data Science in Pediatric Cancer Research | Hubert Hickman | Data Science Hangout

ADD THE DATA SCIENCE HANGOUT TO YOUR CALENDAR HERE: https://pos.it/dsh - All are welcome! We’d love to see you! This week’s guest was Hubert Hickman, Senior Data Strategist and Data Operations Lead at the University of Chicago’s Data for the Common Good (D4CG)! Some topics covered in this week’s Hangout were building data solutions by staying close to the people who use them, harmonizing multi-center research data, working with rare disease data where datasets are small, and navigating a decades-long career across 16 programming languages and constant technological change. One community member asked: “What differences do you see between developing tools for a startup or for-profit organization versus developing things for the public good? Does that change how you think about who the customer is or what the focus of the product is?” (Note from Libby - what a fantastic question 🥹) Hubert’s paraphrased answer: In academic research you don’t have a product focus in the same way, but you still have customers, they’re just not outside paying customers, so you still have to pay attention to their needs. The key factor is distance. When you’re close to the people who contribute, manage, and use the data, like we are at the PCDC (Pediatric Cancer Data Commons), you build things that actually work for them. The further you get from the people who lay hands on your software, the more things tend to unravel. In large organizations you can lose the plot, where you’re 30 steps away just marking off tickets, and if the software is clunky people will use it anyway. That’s not an environment I could live in for very long. Resources mentioned in the video and chat: HamClock Launcher (Hubert’s open-source ham radio side project) → https://github.com/huberthickman/HamClockLauncher R for Data Science (free online book) → https://r4ds.hadley.nz/ Introduction to Statistical Learning (ISLR) → https://www.statlearning.com/ Data Science Learning Community (DSLC) → https://dslc.io/ The Epidemiologist R Handbook → https://epirhandbook.com/en/ Tidy Text Mining with R book (Julia Silge) → https://www.tidytextmining.com/ R Graph Gallery → https://r-graph-gallery.com/ TidyTuesday (open datasets) → https://github.com/rfordatascience/tidytuesday YaRrr! The Pirate’s Guide to R → https://nathanieldphillips-yarrr.share.connect.posit.cloud/ Statistics Globe YouTube Channel → https://www.youtube.com/@StatisticsGlobe Riffomonas YouTube Channel → https://www.youtube.com/@Riffomonas Statistics 1 with R (Leiden) → https://poweleiden.github.io/statistics1/ “The Great Awakening of Agile Engineering” (Neal Richardson blog) → https://enpiar.com/2026/07/06/the-great-awakening-of-agile-engineering/ posit::conf 2026 Registration → https://conf.posit.co/2026/ ► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu Follow Us Here: Website: https://www.posit.co Hangout: https://pos.it/dsh The Lab: https://pos.it/dslab LinkedIn: https://www.linkedin.com/company/posit-software Bluesky: https://bsky.app/profile/posit.co Thanks for hanging out with us! Timestamps: 00:00 Introduction 07:32 “Could you tell me a little bit about how you met that friend? Did you join a community like this one? What did that look like?” 09:34 “I’m wondering what advice you would give somebody who is coming in new to the game and is being tasked with building a data solution for somebody?” 11:30 “I’m wondering if you have seen AI-driven development shift the building-bespoke-stuff game a little bit?” 14:08 “What was your favorite programming language and your least favorite programming language, and why?” 16:55 “Are R and Python the languages that you prefer for data science more specifically, versus other software engineering tasks and things of that nature?” 19:31 “What does an average day look like for you? Or maybe an average week, if every day is not the same?” 22:47 “What differences would you think of or see between developing tools for a startup or a for-profit organization, versus developing things that you’re doing for the public good?” 26:33 “What was your biggest struggle starting a company?” 30:12 “Any introduction to R classes or resources that people would recommend to someone who is just starting out?” 33:22 “What types of problems in cancer research presently most pique your curiosity from a math/stats perspective, and how do you think about approaching them with data?” 37:10 “Is there an infrastructure you use when working with collaborative multi-center data without actually sharing the data, for example DataSHIELD, or another way for federated data?” 43:40 “I was wondering if you would like to share at all about your side projects, specifically your ham radio ham clock launcher?” 47:39 “Do you have a piece of career advice for us? Something that you wish you had been told, or something that you have just found helpful for you over your career?”

Julia Silge, Neal Richardson

Building Reproducible LLM Workflows | Leslie Emery | Data Science Hangout

ADD THE DATA SCIENCE HANGOUT TO YOUR CALENDAR HERE: https://pos.it/dsh - All are welcome! We’d love to see you! This week’s guest was Leslie Emery, Senior Principal Data Scientist at Bristol-Myers Squibb! Some topics covered in this week’s Hangout were data harmonization in Pharma, using LLMs for reproducible data workflows (you read that right!), transitioning from academia to industry, and building custom Shiny apps for internal stakeholders. One community member asked: “Given your role at BMS, what are some of the challenging aspects and rewards of your role?” Leslie answered: The most rewarding thing is working on something that someone else is specifically asking for and waiting for, and they’ll be excited to use it the moment it’s ready. Unlike academia where you might work for months or years and publish a paper without hearing feedback, in industry you get immediate feedback helping colleagues who need analysis-ready datasets. The challenging aspect is that BMS is an enormous organization, so everything requires collaboration. She had to learn that instead of trying to learn every single thing herself (the academia mindset), she should look for colleagues who already have expertise and collaborate with them. Resources mentioned in the video and chat: OHDSI Data Standardization → https://www.ohdsi.org/data-standardization/ CDISC Standards → https://www.cdisc.org/ FHIR Healthcare Data Standard → https://fhir.org/ tidyCDISC R Package → https://github.com/Biogen-Inc/tidyCDISC ADaM Standard → https://www.cdisc.org/standards/foundational/adam XKCD Standards Comic → https://xkcd.com/927/ Book of OHDSI → https://ohdsi.github.io/TheBookOfOhdsi/ Posit Connect Cloud → https://connect.posit.cloud/ Posit Data Science Lab → https://pos.it/dslab Positron IDE → https://posit.co/products/ide/positron Plumber for R APIs → https://www.rplumber.io/ Plumber2 → https://plumber2.posit.co/ Data-as-a-Product Talk (Clara Amorosi, BMS) → https://youtu.be/RrBdbbinS1E?si=gS3JCp0dqFFUVeZx PCDC Data Dictionaries → https://commons.cri.uchicago.edu/data-dictionaries/ Proteomic Data Commons → https://proteomic.datacommons.cancer.gov/pdc/ Cancer Data Aggregator → https://datacommons.cancer.gov/ Ravelry Knitting Community → https://www.ravelry.com/ LibraryThing Book Tracking → https://www.librarything.com/index.php Climate Change Data Visualization in Knitting → https://www.smithsonianmag.com/smart-news/knitting-enthusiasts-are-using-their-craft-visualize-climate-change-180974231/ Quinn Daedal Fabric Visualizations → https://quinndaedal.com/blog/2023/12/21/locked-loom-1-gideon-the-ninth/ ► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu Follow Us Here: Website: https://www.posit.co Hangout: https://pos.it/dsh The Lab: https://pos.it/dslab LinkedIn: https://www.linkedin.com/company/posit-software Bluesky: https://bsky.app/profile/posit.co Thanks for hanging out with us! Timestamps: 00:00 Introduction 2:08 “What do you like to do for fun?” 3:23 “What types of data do you see and problems do you solve?” 9:10 “What are some challenging aspects and rewards of your role at the intersection of multiple domains?” 12:25 “What LLMs have you looked into for data harmonization?” 14:52 “Did you knit your sweater?” 15:18 “What sci-fi books do you recommend, and do you think AI could help or hurt with data harmonization?” 20:11 “Does BMS use data curation like CDISC or OHDSI?” 23:08 “How do you get your company to put resources into changing computational infrastructure?” 26:17 “What tools does your team use?” 29:16 “How often do you have to say no to harmonizing incompatible datasets?” 32:40 “Have you figured out how to use ontologies and data dictionaries in your harmonization flow?” 38:08 “Can you share advice for getting into work at the intersection of technical knowledge and data science?” 44:02 “Are you incorporating skills or MCP for providing necessary context?” 45:23 “Is there a way to measure levels of harmonization or dissonance in data, like in music?” 47:39 “Do you still have long-term projects? How do you motivate yourself through them?” 50:54 “Do you have a Ravelry account for tracking knitting projects?”

Analyzing a TidyTuesday dataset with Posit Assistant in RStudio | Hadley Wickham | Data Science Lab

The Data Science Lab is a live weekly call. Register at pos.it/dslab! Discord invites go out each week on lives calls. We’d love to have you! The Lab is an open, messy space for learning and asking questions. Think of it like pair coding with a friend or two. Learn something new, and share what you know to help others grow. On this call, Libby Heeren is joined by Hadley Wickham who uses Posit’s new Posit Assistant (not to be confused with Positron Assistant or Databot in Positron) in RStudio to analyze the week’s TidyTuesday dataset. Hosting crew from Posit: Libby Heeren, Isabella Velasquez, Hadley Wickham Hadley’s Blueskey: https://bsky.app/profile/hadley.nz Hadley’s LinkedIn: https://www.linkedin.com/in/hadleywickham/ Hadley’s GitHub: https://github.com/hadley Hadley’s website: https://hadley.nz/ Resources mentioned in the video and chat: Posit AI / Posit Assistant waitlist → https://posit.co/products/ai/ Tidy Tuesday Dataset → https://github.com/rfordatascience/tidytuesday/blob/main/data/2026/2026-02-17/readme.md#agricultural-production-statistics-in-new-zealand nanoparquet (R package) → https://nanoparquet.r-lib.org/ Databot → https://positron.posit.co/databot.html wbstats (R package for WorldBank API) → https://github.com/gshs-ornl/wbstats World Bank NZ data → https://data.worldbank.org/indicator/SP.POP.TOTL?locations=NZ Hadley & Joe’s posit::conf talk ft. Hadley’s demo of ggbot2 → https://www.youtube.com/watch?v=tSFHQWGyRzo Wikimedia: New Zealand Total Population → https://commons.wikimedia.org/wiki/Data:New_Zealand_Total_Population.tab Posit Conf Talk: James Wade’s “Disposable Shiny Apps” → https://www.youtube.com/watch?v=smnrmTtoiOM Hadley’s unhinged {genzplyr} (R package) →https://github.com/hadley/genzplyr Simon Couch’s blog post on evaluating LLMs → https://posit.co/blog/python-llm-evaluation/ Garrick Aden-Buie mentioned working on a collection of skills → https://github.com/posit-dev/skills Max Woolf’s blog post “Does Offering ChatGPT a Tip Cause it to Generate Better Text?” → https://minimaxir.com/2024/02/chatgpt-tips-analysis/ Cat Hicks’s skill for coding agents that lets you learn from the agent → https://github.com/DrCatHicks/learning-opportunities (but also see her great thread on bluesky! https://bsky.app/profile/grimalkina.bsky.social/post/3mevvbm3a6s26) ► Subscribe to Our Channel Here: https://bit.ly/2TzgcOu Follow Us Here: Website: https://www.posit.co Hangout: https://pos.it/dsh The Lab: https://pos.it/dslab LinkedIn: https://www.linkedin.com/company/posit-software Bluesky: https://bsky.app/profile/posit.co Thanks for hanging out with us! Timestamps of Questions / Topics 00:00 Introduction 01:17 Posit Assistant and using an agents.md file 02:54 Downloading TidyTuesday data using Posit Assistant 04:38 “Why use Parquet files over CSVs?” 07:11 Oops, switching Hadley’s mic, sorry! 09:27 Creating a visualization of sheep populations 10:28 “When would you switch models to Opus?” 13:28 “Do we have a Gen-Z human in the loop to verify the syntax?” (We did, folks, we did) 13:54 “How many bow ties does Hadley own?” 14:24 Asking the AI to retrieve real New Zealand population data 15:25 “Can the Posit Assistant use voice to text so people don’t have to type…?” 18:40 “What’s the purpose of the agents.md file and what goes in there?” 20:59 “How does posit AI in RStudio compare with the current positron AI assistants?” 22:39 “Will all the code generated by Posit assistant be stored in the session somehow to let somebody review what the agent generated?” 26:08 Having the AI create a Quarto report 26:54 “Does Posit Assistant also support Skills Files?” 27:49 “Were we always able to add agents.md in Positron too?” 28:17 “Did Hadley choose the name agents for agents.md or does each session have an agents file similar to having a README.md?” 29:57 Fixing the AI-chosen plot theme 31:55 “Is any of the stuff that it’s put in the report in Gen-Z speak, or is it only what it’s returning to you in the chat?” 33:24 “Can Posit Assistant create a renv as well?” 35:39 “Is Claude also the best for Python code?” 36:20 “If you add a behavior or a requirement to agents.md, does it take that new info in on the fly as you continue prompting?” 37:40 “What if you ask the agent why something happened?” 40:31 Investigating AI speculations about dairy cattle numbers 41:15 “Do you still need to know how to code in 2026?” 41:42 “Does using caps lock increase the likelihood that the LLM is going to listen to your agents.md file?” 45:47 “Could Hadley have done this faster without an AI?” 47:26 Hadley had AI do text clustering on image descriptions (previous TidyTuesday dataset) 48:41 Hadley had AI make a Shiny app to explore cluster sizes (previous TidyTuesday dataset) 52:53 “If one of the best ways to learn how to read code is to write code, what’s gonna happen if we’re writing it less?” 54:05 Wrap up and Posit AI waitlist information

Garrick Aden-Buie, Hadley Wickham, Simon Couch

Reproducible Examples with the reprex package

Reproducible Examples and the reprex package. https://speakerdeck.com/jennybc/reprex-reproducible-examples-with-r Jump to: 0:08 Intro 0:40 Basic usage of reprex 3:35 Motivation, why use reprex? “Help me help you” 4:08 Define reprex? Three commons ways to use the term. noun, a reproducible example the reprex package. a tool to build R reprexs reprex::reprex(), a function in reprex to make a reprex. 5:26 When should you use a reprex? 6:14 reprex installation and setup. How do you actually get repex on your machine? 7:59 Advanced setup and discussion. 9:45 Please use advanced features responsibly. 11:02 Why does the reprex package exist? Anyone who has helped teach R or dealt with github issues, twitter, stack overflow & RStudio community questions knows that helping people diagnose their coding problems can be hard. This tool comes from hard-won experience. It’s aim to is help people ask well formed questions and increase the chances of getting well formed answers quickly. 12:52 philosophy behind reprex code that I can run code that I don’t need to run code that I can easily run 13:52 code that I can run. 17:25 Tips on writing good reprexs. Dos and don’ts. 18:52 How do I get my data into my reprex? Getting small data and CSV type data into your reprex is easy. “I have a big hairy data object and I can only show their problem by using it”, but that’s not always the case. 21:02 code that I don’t need to run reprex gives your reader the code and reveals the output being produced by that code. For experienced coders, that might be enough to help you. 22:44 code that I can easily run Don’t copy and paste from the R console. This is usually annoying for your reader. Worse than console copy-pasta is the screenshot. (Many people think screenshots of code are downright offensive.) 25:03 reprex_clean If you copy someone else’s reprex into your consolve, it may include their output, making your new reprex a untidy. Here are tips for taking someone else’s reprex code and output, and create a clean reprex reply. 25:54 shock and awe More interesting features of the reprex package. 26:29 What about figures and plots in your reprex? So happy you asked about that. reprex will automatically upload your images to imgur.com. 28:23 Create a reprex by explicitly providing your code in the reprex call. 29:00 when you need your reprex to work in the current working directory. 30:45 Differently flavored markdown. Optimize your reprex markdown output for github, stack overflow, or the RStudio community. 30:31 Make your reprex create an R script, with your reprex outputs as comments. This is handy for pasting into an email or slack-type-app. 32:25 Rich text format, rtf output. (currently experimental feature as of this video) 33:06 supress the reprex add at the bottom of your reprex 33:19 Include session info. 33:54 Auto styling of your code. Good if you’re dealing with poorly formatting code. 34:25 Change your comments string. 34:32 Silence Tidyverse startup messages. 35:00 Capture a reprex that sends messages to standard output and standard input (e.g. package installation compilation messages). 36:13 Set up personal defaults for your reprex usage. 36:54 reprex RStudio addins; render reprex and reprex selection. These accelerate your use of reprex. 39:01 The human side of reproducible examples. How to ask questions in ways that are most likely to get answered. Sorry for the tough love, but this is important. Why are you always asked to give a reprex? Experts try to use reproducible examples to ensure their advice works. Making a good reprex is hard. But, you are asking them to solve a problem for you, so meet them halfway. Creating reprexes is good coding practice. Making a good reprex is often a good way to debug your issue in the embarrassment-free privacy of your own home. reprexes lead to discussions more likely to help people in the future. 44:34 Behind the scenes of reprex 44:44 Thanks for those that helped make reprex possible. Questions and Answers 46:05 can reprex capture variables and objects in the current environment? (not yet, maybe in development) 47:25 does reprex actually check that the code is self contained? (self contained) 48:08 does readr::read_csv support the text argument? (yep, just read the help manual for readr)