Workflow Demo Live Q&A - July 31st

and Shiny for Python. Thank you all so much for joining us today and thank you to Garrett for such a great session. As a reminder, we host these workflow demos the last Wednesday of every month and they are all recorded if you ever want to go back and check them out.

I'm going to share this link with you all in the chat here if you want to make sure it's added to your calendar monthly. I think there's probably over 16 or 17 different workflows now shared there. So from building a model annotation tool to different pins, workflows, to working with Databricks, and it's a mix of R and Python workflows.

I do just want to add this in the beginning. While I know many people today are current customers, but if you're new to Posit Team and you want to try it out for free or you want to chat more with our team, I always welcome you to feel free to DM me on LinkedIn. I can put it in the chat in just a second. Or if you want to book time to chat with our team one-on-one, I'll share that link with you all in the chat here too. We're always happy to learn more from you about your own use cases and chat with you.

Introductions

I'd love to just go around and do some quick intros here. I know, Garrett, you introduced yourself briefly in the video, in the demo, but you want to get started and introduce yourself here? Sure. Yes. Can you hear me okay? Yep. Sounds great. So I'm Garrett and I work at Posit and I'm a developer relations person. So I'm basically just a developer who writes apps with wonderful tools that my colleagues make. And that's a good segue to Barrett. Hi, I'm Barrett Schlerke. I'm a shiny software engineer and I write wonderful tools to have my colleagues use. Barrett's on the development team for Editable Tables.

Awesome. Thank you both for joining us here. I didn't introduce myself yet, but I'm Rachel, I lead customer marketing here at Posit. And so I host a variety of different community events to bring us all together and share different use cases and connect with each other. So I also host our weekly data science hangout. Would love to have you join us in that as well.

Posit conf workshops

Let's jump in. Or actually real quick. I know Posit conference is coming up so soon. I think we're like 10 days away right now. And I know both Barrett and Garrett are going to be there. So I was just curious, do you want to share with us a little bit about the workshops that you're involved in or leading while we're there?

Sure. I'll go first. I, along with my collaborator, Andre DeVrie, will be teaching a workshop that's an introduction to Shiny for Python. And that's for anybody. It's great if you have a background in R and you want to shift over to using Shiny for Python or maybe use Python and you're curious about Shiny or maybe you're new to both. This would be the workshop to start at.

And Barrett, I know you're TAing one, right? Yes, I'll be a TA for the level up with Shiny for R done by Garrett. And it's a mid-level workshop. So it shouldn't be your workshop for your first Shiny app. We have a different one for that, Intro to Shiny for R. But if you've made a couple apps and you're starting to now worry about database connections or caching or just ways to improve your interface, this is a wonderful workshop to be involved in.

Q&A: editable tables

Well, I know there were a few questions that came in during the demo, and I see some already coming into the chat here. So I just want to let everybody know you can ask questions right in the YouTube chat here, or you can also ask questions anonymously at the link that I just showed on the screen.

Keshav asked a question that was, can we make only one or few particular columns editable? I can take that. That's an easy one. I can take that. Yes, you can. The way to do that is when you write your set underscore patch underscore fn function, you could check which the column index is for the cell that the user tried to edit. If you're writing a statement, if it's a cell or a column that you don't want edited, just maybe use the UI notification to tell them that, or maybe just return the original value. And if you look at the mortgage app GitHub, I think I'm doing that there. I think you can only edit the payments column in that one.

And I see Jeffrey asked a question as well. Just curious if you've had any experience with editable tables that reference one another. I have, and it gets complicated. I suspect Barrett might have more sophisticated experience with this. Yes, you can. It's as long as it flows in a single direction, and there's no cycles or circles, then you can connect as many data frames as you'd like. I've done an example where I had the original data on the left. I could then sort and filter just how I wanted to, then I would take that and display it as an original data in a second data frame. And then I could sort and filter on that. And then I had a third output just to prove that they're disconnected, but there is a chain of command that can happen. So add as many as you want, just as long as there isn't a cycle.

And Barrett, I think that's the example in the shiny blog post we started with, right? Yeah, that was a fun little demo where, yeah, the second table depended on the first one.

I see Philip asked a question during the demo, and it was, why do the rows not disappear once you've edited the NA? That was the way I designed the app. So the filtering step, the way I wrote it, will only react to if you change the text box of what the missing value is. And once you've made that change, the rows of the table that appear go stay there persistently until you make that change again. But if you didn't want that behavior, if you want a different behavior, you could write the app differently so that it's paying attention to both changes in the table and changes in the value on a filter against.

Reactivity and linked tables

So let's go back to that linking question, and I see Bill asked, how does all that linking stuff work? Does Reactive come into play with this? Barrett, maybe you could tell us more about how to make it a Reactive thing to respond to versus, you know, like in the filtering thing, I didn't make it Reactive. Yes. In Python, your outputs are objects at the end, which is great. We can use these objects, and we've added on these extra methods that are explained in that 0.9.0 blog post where we're trying to empower the renderer to have a little bit more context, and so we've added methods like .data or .dataview of what the user is looking at, and those functions or values are Reactive. So if you use them in a Reactive setting, they'll automatically trigger with updates whenever, like, things happen upstream, which is great. And there's other things in there, like cell selection, sort, and filter. You can get all of that information reactively, given what the user is looking at.

I'm not sure if this one is related as well, but I see Chris asked, what would be an example of a cycle that does not work? Yeah, I'll take this one. So let's imagine we have two data frames. We have DFA and DFB. You could define in Python saying that DFA's data is DFB.data, and then you could find, like, define on B that its data comes from DFA, and so they would depend on each other. No input would actually come in, and, you know, your computer explodes or something. I don't know. But you need something to where, like, you have your original data set here. You then come in to A. You then go to B, and then you can go on to C and D and whatnot, but just don't have it loop back.

Comparing editable tables in R vs Python

So I see there's an anonymous question that came in very early into the demo, and it was, why is this not DT for Python, and can you compare and contrast editable tables in R versus editable tables in Python? Is there a reason to reconsider R for this?

DT is a wonderful package. It was the best on the block 10 years ago when we did it, and there's been a lot of advancements in the JavaScript world in the recent years, and so we're trying to leverage that, you know, using TanStack underneath the hood so that things are as fast as possible when rendering. So we, you know, there's some engineering design changes that needed to happen, and so we can't just reskin it under Python. There is packages like ITables, I believe, in Python that is a wrapper around DT, so there are ways to do it. There are some Shiny widgets, I can't remember, that possibly work in... Or not Shiny widgets, IPy widgets that work within Jupyter Notebooks, but the reactive model within Shiny is really backwards when you're using IPy widgets with Shiny, just because IPy widgets pushes you information versus you reacting to that change. I know it's a subtle thing verbally, but in practice, it feels really wonky within the Shiny world. So we just needed to catch up to the R world.

I mean, we've had 10 years of experience in the R side, but my usual answer of like, should I use Python or R? My generic response is usually whatever you're fastest at producing the desired output, you should use that. Maybe there's a better tool over here or over there, but if you're most comfortable like doing your data analysis in R, you should stay there. If you require a certain feature, you know, maybe you switch to Python.

I mean, we've had 10 years of experience in the R side, but my usual answer of like, should I use Python or R? My generic response is usually whatever you're fastest at producing the desired output, you should use that.

It probably is a good time to address this question as well. I know, Garrett, you briefly covered a way to do this with R in the beginning of the demo. So you want to comment on this. Is this functionality available for Shiny for R yet? Yes, it is. And that would be with the DT package that was referenced in the last question. So making an editable table is a feature of the DT package. Maybe you might know better than I if the feature parity is exactly the same or not, but you can edit tables with DT and Shiny for R.

Multi-user editing pitfalls

I see Kelsey asked a question over there. And it was, any pitfalls to watch for if we use this type of app for multiple users editing a data table? We would like to record when they make changes and revert changes. Yes. I'll take this one. So this, there are a lot of pitfalls. I mean, we could have things like when you save to the actual database, I would say transaction would say, and if Garrett and I were doing that at the same time, then one of us would win and the other one would get stomped over, and that's not good for a database transaction. I explicitly did not try to make any forced design decisions. So there's the, as Garrett mentioned earlier, the set patch function where you can receive an edit from someone, and then it's now your responsibility as the app author to handle that. Maybe you're just writing to a local CSV file and it doesn't matter. Maybe you're ignoring it completely or you're all sharing the same database connection. So I would have to default to say like, please leverage your database's ability to perform transactions and put a lock on the system. Yes, that may deter editing, but at least you'll still be able to.

Excel integration and formula support

I see one other question from Bill, which was, one second, can you embed an Excel spreadsheet in this type of workflow using a Python or R package? I'm personally not aware of one, but I think what this is doing is bringing us a step closer to making the table that is in Shiny function like an Excel spreadsheet. And then here I've downloaded to CSV. I don't use spreadsheets that much, but I assume it'd be very easy if I had a package that could export a data frame as an Excel spreadsheet, and then you could just pick it back up in Excel outside of Shiny. I know I've done that before, actually, where you can add a button to the top where the end user can just download the Excel version of the table.

So here we're relying on Pandas's to CSV method to take that data frame and put in a CSV format. And then the download button will take whatever we have and pass it through to the user. So we just need to get into the Excel format. I feel silly that I can't say with certainty whether there's a to Excel method from Pandas. It seems like there would be, but I just don't know. Maybe someone in the chat knows too, if you want to jump in. Barrett, anything to add? From my knowledge, I'm interpreting Excel spreadsheets as formula support. And as far as I know, any editable data frames do not have formula support by default. There is the R package of R hands-on table, I believe, and it looks like a spreadsheet, but that's about it. It's very nice editing view, similar to DT, or a little bit prettier than DT, but the formula support isn't there.

The door has been opened in the Python side for PyShiny, Shiny for Python, that if you edit a cell, we can actually edit more cells as a reaction to that. So if you change the value for one cell, we can change the value for multiple cells, which I think is really neat, but that is not automatically handled. I'm just kind of putting it out there to see if anyone starts to leverage it with formulas.

GT tables and Excel export

This is shifting gears a little bit from the demo, but I know there was a very long thread on LinkedIn. Melissa had posted something out to the community asking if there was a clever way to programmatically export a GT table to an Excel file, and it seems like these Excel examples might be good to double down on for a few people here as well.

I'll go with the question is more for the RGT package, and GT is like the absolute king of styling your data frame data table. It's the best because you can compose everything. You don't have to have one function to make all the design choices. It is absolutely awesome package to use, and I know they can export to CSV, and CSV can be read into Excel, but that loses the color styling or the bold italic font styling, things like that, which is unfortunate. Rich has the most focused energy that I've ever seen in someone, so if he has not addressed it, and it's one of those early issues, there's got to be a real limiting factor because I know he's there for the user.

Posit conf workshop materials and closing

I did just share the GT package for R and then also Great Tables, which is a Python version of the package as well, if anybody wants to check those out too. Oh, there is one question I missed that's around PositConf. So Talon asked, I saw in the FAQ that the workshop materials will be posted a week afterwards on GitHub. Will they be posted under the Posit GitHub? From past experience, Garrett or Barrett, do you know? Yes, they will be posted. I don't think they're posted under PositDev, which is the GitHub we make software. I believe it will be a conference organization.

And I imagine that this will be communicated on the same websites we're currently communicating about PositConf. While you're grabbing that, I just want to remind everybody that PositConf also has a virtual aspect of it as well. So even if you can't join us in person in Seattle, we'd love to have you join us virtually too. Posit-conf-2024 will have all the workshops.

Oh, Chris said, I'd like more Excel demos. This type of input gets close but misses the mark for what I want. And Chris, I would love to learn more from you about what you'd like to see. And if anybody has specific examples of what they're trying to do or workflows you'd like to see in these monthly sessions, you can email me directly. It's just rachel.posit.co. But we are creating these examples because we want them to be helpful to you all and achieve workflows that you're trying to make happen. So if you want to add comments into the YouTube chat here for things you'd like to see or if you want to email me directly, that's awesome.

But thank you all so much for joining us today. As a reminder, these are monthly, so it's the last Wednesday of every month at 11 a.m. Eastern time. Garrett and Garrett, any last parting thoughts? Anything you're really excited about as you think about the future of these workflows and packages? I'm interested in trying out that changing multicell thing, see how close we get to formulas. It'll get you pretty far. We just need to set up some rules.

I just want to add that the editable data frames are rapidly evolving. You know, we're listening to you. Please do make GitHub issues or, you know, email Rachel. Like, any way you can get that desires towards us is helpful. When we first came out with data frames, I think within two weeks, we had a request for editable cells. I think that took, like, five years on the R side. So it's really neat to see how it changes within the Python world. And yeah, we are rapidly improving. It's not done yet. The story isn't done. So please stay tuned to the advancements coming soon.

When we first came out with data frames, I think within two weeks, we had a request for editable cells. I think that took, like, five years on the R side. So it's really neat to see how it changes within the Python world.

Awesome. Well, thank you all so much. Hopefully I'll get to see many of you in person in Seattle for the conference. But if not, I'll see you back here in a week.