Coding Tools for Industry R&D – Development Lessons from an Analytical Lab - posit::conf(2023)
Presented by Camila Saez Cabezas Are you considering or curious about developing code-based tools for scientists? Whether you are an experienced developer or a fellow Posit Academy graduate who might be stepping into this role for the first time, the aim of my story is to inspire you and help you navigate this process. While developing custom R functions, packages, and Shiny apps for diverse analytical capabilities and users in R&D, I learned why it's important to collect certain information at the start before writing any tidying, analysis, visualization, and web application code. In this talk, I will share the essential technical questions that help me define and plan for success. Presented at Posit Conference, between Sept 19-20 2023, Learn more at posit.co/conference. -------------------------- Talk Track: Lightning talks. Session Code: TALK-1168
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hello, my name is Camila Saez-Cabezas and I am a scientist in R&D at Dow, where I deliver analytical solutions using my expertise in measurement science and R programming.
Have you ever been eager to jump into a coding project intended to help others, but then were disappointed by the development process? This was me when I first started coding tools for my fellow R&D scientists who are not programmers. I suspect I'm not alone. Do you remember what your biggest source of frustration was? For me it was the maddening back and forth between feature requests, people changing their minds about the final product, and rewriting code under a tight deadline in addition to unexpected technical challenges.
We can't control the unexpected, however, by controlling our urge to code prematurely, we can improve the impact, efficiency, and experience of our development process. My goal is to share the lessons I've benefited from and convince you to invest time and energy into non-coding tasks. Don't worry, let's tackle this at the beginning to avoid major course corrections when we're deeply engaged in coding, demos, deployment, and implementation.
We can't control the unexpected, however, by controlling our urge to code prematurely, we can improve the impact, efficiency, and experience of our development process.
I learned to dig deeper during the brainstorming and data inspection phases and gain essential technical information by playing three roles. I will demonstrate why using the analytical R&D lab as an example, but I believe this approach is useful across many areas and scenarios.
The mission of analytical scientists in R&D is to problem-solve diverse materials and complex chemistries by interpreting and combining large volumes of data from different instruments. Scientists typically adapt their data workflows to commercial software, not the other way around, which is why they need your expertise to dream, design, and execute the vision for their custom tools. Specifically, they need guidance on how to extract more value from their measurement data with code and what infrastructure packages and resources suit their needs.
Let's step away from Posit Workbench temporarily, visit collaborators in their natural habitat, and explore together.
Be a detective
Your first assignment is to be a detective. Shadow your users to discover their sources of pain points and inefficiencies. By asking questions as they load samples, run measurements, and analyze data, you will identify clues and assumptions that are overlooked when these steps are simply described to you. For example, you notice that your users can export data in a different format that is easier to parse, or that it's possible to save time and avoid copy-pasting errors by extracting instrument metadata from a file otherwise ignored.
Be an architect
Next, be an architect. Present your discoveries to the team on the drawing board, help them create the big picture, and map the path forward. Brace yourself to negotiate structural and aesthetic preferences and balance time frame, cost, and risk constraints. Another potential challenge is getting users on board with the solutions they need instead of replicating their favorite software features.
Maybe your users are fixated on how they like to edit and annotate graphs in Excel because they haven't grasped the benefits of reproducible and interactive plots yet. Maybe you've been tasked with building a web app, but you've identified that what they need is an automated and scheduled report. My point is, here's your chance to pitch your approach and influence the foundation of the project in a rewarding way.
Be a chef
Finally, as a chef, choose the perfect ingredients, how to season, and prepare them. Think of it as planning ahead for a big demo dinner party and anticipating the right amount of food to cook to keep your project on track. Imagine people asking, can we test more datasets to increase our confidence? Did we account for both PDF and CSV files? However, it might be too late or tricky to incorporate these changes.
My intention is not to dismiss the iteration process or testing new ideas as a project evolves. Instead, agreeing on the representative datasets and require performance early will set the stage for a sweet and satisfying recipe for success.
Instead, agreeing on the representative datasets and require performance early will set the stage for a sweet and satisfying recipe for success.
I hope that I've inspired you to engage with your stakeholders and collaborators as a detective, architect, and chef. Try it for your next project and reach out to discuss more. I love brainstorming in this space, and it's better when we do it together. Thank you for listening.