Workflow Demo Live Q&A - August 28th!

Okay, well, thank you all so much for joining us today and thank you to Isabella for such a great session.

As a reminder, we host these Workflow Demos the last Wednesday of every month and they are all recorded as well. So I'll share the link in the chat right now if you want to make sure they're on your calendar for the last Wednesday of every month. I think it's about 17 different workflows now shared to the YouTube playlist. So let me put this playlist here as well.

But I also know a lot of people who are joining us today are current customers, but if you are new to Posit Team and are curious to try it out for free or if you want to chat more with our team in greater detail than we can cover in the Q&A, feel free to DM me on LinkedIn or you can always book time directly with us here. And those are my few links. I'll just share in the chat as we get started.

But I'd love to just quickly go around. I know, Isabella, you introduced yourself for the demo, but just do some quick introductions here as well for the Q&A. Isabella, you want to get started?

Sure. Hi, everyone. I'm Isabella Velasquez. I am on the Developer Relations Team here at Posit. Coming up on my three-year anniversary, I love working with R and Python and creating Shiny apps, and I hope that you enjoy today's webinar. Thank you for joining.

Thank you. Ryan, are you frozen or you want to introduce yourself?

Oh, let's do a test. Can you hear me okay? Okay, there we go. Yes, we're good.

Okay, I apologize. It seems like my internet and my co-working space seems to be on the fritz. But yeah, I'm Ryan Johnson. I'm a Data Science Advisor here at Posit. I've been here for a little over four, four and a half years, I think. I work a lot with our customer success and sales teams, making sure that folks that are using our professional tools are getting the most value out of them. And so a lot of times that means using our really cool open-source tools like Pins. And I've hosted a few of the other sessions in the past as well. So I use Pins a lot in my various workflows. Hopefully, I can answer your various questions.

And I'll introduce myself as well. I'm Rachel Dempsey. I lead Customer Marketing here at Posit. And so I love getting customers in the community together in different events like this one, or our Data Science Hangout that we have every Thursday. So it's so nice to see you all today. Let's jump into some questions.

So I'm gonna, oh, just to remind you how you can ask questions as well. I can see people are already asking in the YouTube chat here, but you can also ask questions anonymously if you want. And this is the short link. I'll just put in the chat here and show it on the screen where you can ask on Slido.

Pins reactive read and board types

So I see a question that came in in the chat just now, which was, does the PIN Reactive Read detect changes because the Shiny app and PIN are both on the same Connect server?

I believe the answer is no. Ryan, correct me if I'm wrong. PINs can be shared on a variety of boards, such as AWS or Google. And like the PIN Reactive Read is, you know, works regardless of the type of board. Welcome to your thoughts.

Yeah, no, that's exactly right. That's my understanding as well.

Using Python and Quarto together

A question that came in on Slido towards the beginning was, is there any other efficient way to code in Python without using, this sounds funny to read this question out, but Python. So I think it's the way it was written in the Quarto doc, but Isabella, do you wanna touch on that?

Yeah, that's right. So in the webinar, I used a Quarto document for the loading of packages, the pulling of the API data, the cleaning it up, and then writing it to a PIN. And I used a Quarto document specifically because I had written the API data pull function in Python, and I like to clean my data in R, and it just did not feel like switching the code chunk from Python to R itself. And so within a Quarto document, or if you're familiar with R Markdown documents, the executable code is written in these code chunks with three backticks, and then the name of the language in the curly braces, then your code, and then ending it with three backticks.

And so anytime in those kinds of documents, if you want executable code, they have to be within something like that, or there's also inline code, but that's slightly different. And so in the example, like if I had chosen to do the entire data pull, say in Python, I could have just used a Python script, like .py, and then uploaded that to Connect and scheduled it for refreshes, or similarly, just written everything in R and uploaded that to Connect in an R script, and then rescheduled the refreshes. The only reason that you see those code chunks in a Quarto document is because I wanted to mix the two languages.

Changing pin names on Posit Connect

I see Christophe had asked a question a little bit earlier, and I see Jeffrey had shared some thoughts in YouTube in the chat as well, but it was, can I change the name of my pin in Posit Connect so that it does not start with my username? I wanna refer to my pin as app underscore name, but I've only been able to get my username.

I have to admit, I have not tried this before. I can see that Jeffrey has shared in the chat some thoughts too, so thank you, but Jeffrey said the pin name can be changed in the info tab of the pin on Connect, but he didn't know that you could change the parentheses name, though, as that's the path that it reads from.

Yeah, and I'll add a, and again, interrupt me if my internet's getting all wonky, but naming things is very hard, and Pins is no exception to this. We actually hosted a workshop at conference where we did leverage Pins pretty heavily, and essentially every single person had their own pin. It was the exact same dataset, but they all had their own pin, and automatically it will append their, or preface their username to that pin name, and so when it's hosted on Connect, I mean, when you pin something, that is your piece of content on that Connect server, and Connect has a lot of advantages of being able to control access to other users, and so that is your, you are the owner of that content, so this concept of being able to change the name, it's not generally advisable, but we'd love to kind of learn about your use case on why you would wanna change the name, and so the Pins developer team, I spoke with Julia Silgi on this issue a lot, and we know we're not quite where we wanna be with the naming of Pins, and so if you have good use cases for why we might wanna change them, or make them more, less personal, so to speak, we'd love to hear that feedback.

Parquet support and large data

Okay, copying a few questions over from Slido real quick. I see Ethan had asked a question that was, does PinRead support parquet files? Wait, hold on. Does PinRead support parquet files, predicate, push down? Say if I have a big data file, but a dashboard only looks at a portion of that file, you don't need Shiny to read the whole file.

Yeah, I can start tackling that one, and Isabelle, you can certainly add in any other context, but so first off, Pins does support parquet files, so that's just something we wanna make sure everyone's aware of, and parquet files are a fantastic way to leverage large data sets very quickly. Parquet's really awesome. Now, if I understand the question correctly, it sounds like maybe you have some large data set that's in a parquet file, potentially pinned somewhere, and you only wanna use a portion of that pin within your dashboard, and not necessarily have to read in the entire thing.

So from my understanding, I don't believe there's a way to read in a pin, and only read in a subset of it. For things like that, I know we really think of pins as like, kind of like a minimal, viable data set or object, it doesn't have to be just a data set, that can be easily read and easily shared across other users and across other pieces of content, potentially hosted on Connect.

So we don't really want to think of pins as more or less like an alternative to a database, for example. So if you have a very large data set, something that's like gigabytes in size, probably not the best use case for pins, so we would suggest putting it into a database, and then the advantage there is that you can actually, through dplyr, and kind of using dplyr under the hood, when you go to make a read from that database, you don't have to read in the entire thing to your Shiny application, you can actually only pull in a subset of that data. So even it's a really good question, I'd probably say it's a little bit more applicable to databases than actual pins.

So we don't really want to think of pins as more or less like an alternative to a database, for example. So if you have a very large data set, something that's like gigabytes in size, probably not the best use case for pins, so we would suggest putting it into a database.

While we're already talking about Parquet here, I see Eric had a question, if you use the Parquet format for writing a pin, can you take advantage of the high grouping features when interacting with the data?

And if any of these questions are something that we can't address on live here right now, I just want to make sure everyone knows, we're happy to chat one-on-one as well, and so I can share that link again where you can schedule time to chat with our team.

Any thoughts, or do we move this one to a separate call?

I would say these concepts are new to me, but this is very helpful in hearing the interest in Parquet, and perhaps we can explore the use of Parquet with Posit tools in a future webinar.

Pins vs. accessing data on a server

Okay, one of the questions on Slido earlier was, how is this better than just accessing data on a server?

Ryan, I think you wanted to take this one. Yeah, it's another really good question, but I guess my argument would be, my data on a server may look very different from your data on a server versus someone else's data on a server. It probably depends on, for your specific use case. If your data on your server is very clean data, it's very easily read in, it can be easily shared across users, then maybe just leveraging the data on the server could be that solution.

But if you have a database, like a data warehouse where data is kind of dumped in, and you want to essentially pull in some of that data, extract it, clean it, make sure it's always being cleaned, potentially on a schedule using Posit Connect, and then load it either back into the database, or you can load it as a PIN on Posit Connect and make it accessible. And that's a really good use case that I see for PINs. It's for taking, again, those kind of minimal chunks of data or some object, making sure it's always up to date, and securely sharing it with whoever using Posit Connect.

Okay. I would say for this use case, just like the Jira board really does not contain too much information. Really what I care about, there is versioning on PINs, but really what I care about is the most, every morning when I check that that is up to date. And so it really doesn't need the heavy-dutiness of a database schema.

And then one other thing I wanted to note is a nice thing about PINs, is that in addition to data, it can store other sorts of formats, so like models, other kinds of R objects as well. We didn't go too deep into that into this webinar, but just know there are other kind of use cases for PINs as well. Highly recommend looking through the website. It has really great documentation.

And I just saw that Jeffrey posted in the chat of really, I know a lot of times when I think of PINs, I think of what Jeffrey just mentioned there. So it looks like he uses PINs for just small things that may not warrant using a database or setting up an entire SharePoint workflow. But again, small things that probably need to be accessed by multiple users or multiple pieces of content.

Data size guidance for pins

So I think this, I mean, I know we just touched on this here as well, but I just wanna make sure for this person that asked this question, if there's anything else we should add here. So what guidance can you provide regarding small or medium sized data? What would be the size of a dataset for which this demo would not be appropriate?

Yeah, it's a good question. And I, so I work like in our solutions engineering team, and we kind of have this arbitrary threshold that we typically tell customers. We say that if your data is getting on the size of about a gigabyte, probably not a good idea to put it into a PIN. That's not to say a PIN can't handle something larger than a gigabyte in size, just for what PINs was designed to do. And a lot of the testing that we've done, it's on things that are significantly smaller in size. So we, again, arbitrary, but we would say a gig, less than that, probably totally fine for a PIN. Once you start getting over a gig, you might wanna start exploring some other options.

Learning about reactivity

One question from earlier as well was, where can I learn more about reactivity?

Yeah, so reactivity is one of these foundational concepts of Shiny. Two resources that I found were really helpful were the Mastering Shiny book, which is freely available online, also in print as well. And then the Shiny website actually has a lot of really in-depth articles on understanding reactivity and highly recommend taking a look. It's like, I think the articles are really valuable resources for this.

Pin scoping and access control

Scott just asked a question here on YouTube, and it was, how are PINs scoped per user, per app, per group? Can the scoping be specified?

Do you wanna chat about that as well, or do you want me to kind of take that one? Do you feel, yeah, you can go first, thanks. I can touch upon it.

So PINs, the nice thing about PINs is that it's a tool that's meant to be fairly, not just language agnostic, so you can actually, there's a PINs equivalent in Python as well, but also kind of where you want to pin things to. So within PINs, there's this concept of a board, and a board could be a variety of things. Now, within POSIT's professional tool chain, we really advocate for the use of POSIT Connect as that board, and it provides a lot of advantages because once it's pinned to POSIT Connect, it automatically is versioned. So if you have new versions of that PIN, you can just keep pinning to the same location over and over again, and access previous versions. But to speak to this question, when it's hosted on POSIT Connect, Connect has a really easy way to scope that PIN to a potential individual person. So if you wanna share it with just one other person, you can do that. If you wanna share it with a group of individuals, so that group can be set up manually via Connect, or your administrators, or you can set up through your authentication that Connect is leveraging.

So, and it's really kind of up to you in terms of who you wanna share it with, or what content you wanna share with on Connect. So you have full control over that. But again, you can pin to other locations, and depending on where you pin that, if it's like some shared network drive internally that you have, that PIN will be essentially scoped to wherever it's living. So it's really kind of up to you. So PINs is pretty flexible in that manner.

LLMs, GitHub Copilot, and Shiny

I'm not sure this one is related directly to the demo today, but I figure I'll go ahead and ask it. Any update on a GitHub copilot or similar integration? Can LLMs be hosted in PINs to provide responses to reactive user inputs?

Yeah, really, really good question. So I guess I'll tackle the first part. And Isabella or Rachel, feel free to interrupt me at any point. But we do have GitHub copilot integration within RStudio on Posit Workbench. So this is something that you're looking for. We do have it integrated with RStudio, again, on Posit Workbench. So that's something that, you can always reach out to us and we can talk more about.

Can LLMs be hosted in PINs to provide responses to reactive user inputs? I would say yes, asterisk. So, you know, with PINs, if it's an R object, you can pin it. Now, I would probably argue that it might be better to put something like this into like an R function, where there's a function matrix where the function may take like some text and then it spits out a response from an LLM. And so functions can actually absolutely be kind of, you know, served potentially through an API. That API can also be, you know, leveraged kind of using PINs in the background as well. So like, I think yes, the answer is yes, we can configure it. I'm not, I do not know for certain if an LLM object can be a PIN. That's something I'll have to probably take a little bit more of a, back to our dev team.

Thank you for the great questions here too. It's always interesting to see like what comes up as well. So we know what we can bring back to the team too. Yeah, this question is super fascinating. I would have never thought of it. And just one thing, like I don't know how much it relates to the person's actual, the person who asked the questions, the actual interest, but just so you know, Joe Ching during Paws.com and Winston Ching as well, showcased like some really cool integration with Shiny and LLMs. And if you registered for Paws.com, you can watch the recordings on the event portal. And then they'll also be on YouTube in a couple months. Again, not related to PINs specifically, but I know there's a lot of exciting things happening with LLMs and Shiny as well.

Board URL and alternatives to pins

Eric asked a question in YouTube chat, which was, I'm interested in using the board URL as a board for custom projects. Are there specific requirements for the type of web platform or hosting for a web-based board?

I would just say that I don't have a ton of experience working with board URL. We actually, we dabbled with it when we were creating the, our content for our workshop at conference, but ultimately just went with the typical board RS connect function. So I guess I just probably don't have enough experience with the board URL function to answer that question, but that's something we can also bring back to the dev team.

Part of what I miss from being in a Zoom conversation with these more forward conversation demos is like getting to hear more from people about their specific use case and the things they want to see. So thank you for all the great questions. And if you want to provide more context to us as well, or feedback that we can share with the dev team, please feel free to even just email me as well with that.

I think we've covered everything on Slido here. Let me go double check.

I think this isn't a question that was asked directly, but I think it is important to address as well. So we talked a little bit about some teams use databases. What would people use if they don't use pins for this?

In terms of the Shiny function? Yeah.

So there are a few other kinds of functions from Shiny that can kind of pull a data source or place and see whether it's changed and then update the Shiny app accordingly. So one example is reactive poll. And I believe there's a called reactive file input or something similar to that. But that's the same sort of like idea as a pin reactive read, checking to see like, has something changed based on a time interval and then showing that on the Shiny app.

Closing remarks

Okay. Now let me just jump over to the YouTube chat. I'm checking in a bunch of places to see where any questions were asked, but I think we have covered everything. I wanted to remind everybody that Isabella shared her GitHub repo for this as well. So you can follow along, take it, make it your own and try it out as well. But thank you so much, Isabella, for sharing this workflow with all of us. And thank you all for the great questions too. I did put into the chat, I would love to hear your feedback on any workflows that you feel might be missing from what we've shown so far. And so even after the fact, if you're watching this as a recording, you can put that below in the comment section. We'd love to hear from you.

And thank you all again. Anything else you want to add Ryan or Isabella? Thanks everyone. Thank you, Rachel. And thank you, Ryan. Thank you.