Kelly O'Briant | Build Your Ideal Showcase of Data Products | RStudio Connect 1.9.0
Posit Connect 1.9.0 - Content Curation Tools Overview Demo / Q&A with Kelly O'Briant As publishers add more content to Posit Connect, content organization, distribution, and discovery can become a challenge. Distributing individual links to all your most important content is tiresome, and the default Connect dashboard contains more information than end-users often want or need. The new 1.9.0 release of Posit Connect introduces tools for addressing these common content curation concerns: How do you make sure your audience finds what they need on Posit Connect without paging through the dashboard, remembering the right search terms, or bookmarking every content item you share? After deploying many pieces of related content, how do you share them as a cohesive project? You might be interested in these content curation tools if you’ve ever wanted to create: * A summary/reference page for a complex project. * A content hub or knowledge repository for work belonging to a team or objective. * A customized entry point into Posit Connect for stakeholders. * A presentation layer for any curated list of notable content items. For more information on Posit Connect: https://www.rstudio.com/products/connect/ If you'd like to do an evaluation: https://www.rstudio.com/products/connect/evaluation/ Blog Post: https://blog.rstudio.com/2021/07/29/rstudio-connect-1-9-0/
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
To kick things off here, at the very end of last month, we released RStudio Connect 1.9 with new features for content curation to build your ideal showcase of data products. And so today, Kelly is going to go through a demo of the new functionality. And we'll be here to answer any and all questions that you have.
So again, please feel free to use the chat window to ask questions. Or I'll put the Slido link in there shortly if there's anything you want to ask anonymously as well. And with that, I will turn it over to you, Kelly.
Awesome. Thanks, Rachel. I'm going to share my screen here. All right. Hopefully that's showing up OK for you.
I can see it. It's not in the presentation mode yet, but I can see the slide. I'm going to be hopping through the browser and in and out of my RStudio IDE. So I'm just going to keep it like this. It doesn't bother everybody. No, that's good. I'm going to present like this. It gives a casual feel. So feel free to ask as many questions as you want.
I also want to give a call out to Katie Maciela, who's on the call as well, who's one of our CS reps. She took some slides that I built and built some better slides. And then I took her slides and turned them into these slides. So Katie, thank you for your slides. I will just keep passing them back and forth and improving upon them forever. And it will be great.
So welcome. I'm here today to talk about RStudio Connect and the release we did last month, which was 1.9.0. It introduces some new functionality and a new R package called Connect Widgets. That is the big headline for this release and R package, actually, not anything having to do with Connect code at all. There's a little bit of functionality that went along with that. I'll talk about some quality of life improvements, we'll call them, to go along with this package as part of the 1.9.0 release.
But largely, this is about an external R package we produced to add some functionality that I think was really missing in the product itself. This is the Connect Widgets official HEX logo, but we haven't officially made stickers of it yet. But Edgar Lees put together this HEX logo himself for us. And so I wanted to give him a special call on his call.
Publisher actions and content management on Connect
But in order to level set here, I want to make sure that everybody is on the same page with us as we talk about Connect and talk about the feature set that Connect Widgets plugs into here. When I think about Connect, I am thinking as a product manager across a large number of different user profiles. So I think about what Connect can do for the admin. I think about what it can do for the data science leader, what it can do for the stakeholders, the viewers. But mostly, because of my background in data science myself, I think about what Connect can do for the publisher.
And when I think about the publisher actions that exist on Connect, I'm sort of thinking along four different vectors, thinking about what Connect can do for deployment. That means deploying different artifacts in R and Python. And you can choose from a number of different deployment mechanisms. You can go push button deployment. You can deploy Python content through a CLI. You can plug into your Git version control system and do Git back deployment. Or you could even fully customize, scope out deployment pipeline using our server API. So you've got lots of different options that take you from the basic, getting something off of your laptop, capturing the basics of that environment, and putting it on to a shared server, all the way to DevOps-style deployments, CICD.
You think about execution. You can choose between publishing static content or publishing content that's backed by the source code itself so that you can set it up to run on a schedule. Or you can control the number of processes and connections to that process if you have an interactive content type, like an application or an API. I think about management. I want to control the metadata that is associated with this individual content item. I want to be able to publish new versions of it in place, like roll them forward, backward, do blue-green-style deployments between different content items.
I want to be able to access the report history if I have a scheduled report and see the history of what I've been producing over time. And I want to be able to do some light organization with tags and track usage widgets. For the final step, distribution, I'm thinking about actually making an impact in my organization. How am I going to get this content that I've spent so much time deploying and crafting and managing into the hands of people who actually need it?
Sometimes it's just you who needs something, or it's you, your future self, who needs to be able to find something. Sometimes it's a group of larger folks who you want to distribute this thing to. You want this to be a showcase of what you're producing for your organization for. And so it's important to be able to make sure they can find that thing on Connect, or make sure you've sent that out and through the proper channels. So you can distribute content that was on Connect through custom emails. You can set vanity URLs. There are a number of things that you can do to distribute individual content items today.
Demo: managing a single content item
So I've produced this report. It's based on this silly toy project that I put together called Cookie Cutter Shop, where I was managing an Etsy shop to sell cookie cutters in the shape of the R logo. I was selling them in three different colors. They were purple, black, and red. And as people were ordering them off of the Etsy shop, my inventory was soaring. And then I would have to update those inventory levels and then start printing more on my 3D printer in each of those colors as the inventory burned down.
So I was producing these inventory burned down charts and updating my inventory as they went up and down. And I was replenishing and orders were coming in. And every day this report would run and then send me an alert or not send an alert based on whether the inventory level crossed a certain low-end threshold that I wanted to trigger the action for taking more 3D printing on.
So this is my inventory control report. And what I can do in Connect, this is the publisher view within Connect. I can control the access settings to it. I can set just me to be able to see it, or I can open it up so that anyone can see it on the server. I can add collaborators. So if I wanted to add Rachel, if she comes on board, I hire her as like, hey, I'm gonna keep track of my inventory now. I should probably add you as a collaborator to this report. So now she can come and manage this report, produce updates to it, check my code and make sure it's correct.
And I can add a link so that we can share this report with other folks. And we don't have to share this like ugly, long URL. We can make a cute, short URL, like a cookie report. Save that. And then we can distribute this whole thing just based off of this individual link.
That's the open solo view. Back to the publisher view. The other thing that's really useful here, as I mentioned, is to have this report running on a schedule. So we want this to run, you know, daily, every day at maybe 6 a.m. That'll be good. 6 a.m., that'll be 9 a.m. for Rachel since I'm on the West Coast, she's on the East Coast. And we'll need to collaborate on this. And then I wanna send a custom email after the update, but I don't want it to send every day. I only want it to send us an email if the inventory falls below a certain level for a certain color.
So I've written that logic using a Plastula package into a custom email associated with this content item. We publish both the report and the email together as a bundle here on Connect. So now when I set this report up to get published and run every day, this will send me the email. I have a little screenshot of it. I won't show you my actual email because that's a disaster. But here's what a report email looks like. I have a little banner image and it lets us know that we need to go check the inventory levels and everything is wonderful.
So those are the basics of managing access, distribution, scheduling for a report-type content item. There are a couple of other things that you can do here that not everybody takes advantage of on the Connect server. One is tagging. If you want to go through and add a level of organization to your Connect server, you can have an admin set up a tag schema for you. And as a publisher, that means that you have access to all of the tags that your tag schema will provide.
So these are hierarchical tags, which means that they work a little interesting. If I want to tag something as Python content, I can then go in and tag it as a subclass of Python content. So I can apply the Python tag and save the dash tag. Now, that doesn't make sense for this particular document since I wrote it in R, but that's just an example of how the hierarchical tag system works. So you'll see that I tagged this as email since it has a custom email associated with it. And this is our sales demo server. You'll be able to find reports that have custom emails associated with them very quickly.
The content discovery problem
And I think this works like really well for single items, at least. If you are in a scenario where today you have Connect and you only have like five different content items and they're all beautiful, you've set vanity URLs for them and they're all tagged, maybe you don't even need tags, you only have five items. So everything is labeled and maybe your stakeholders have to do a little bit of reading. They've read the description, associated metadata and thumbnail images with them. Everything is fairly discoverable. Content discovery is not an issue for you.
But if you're like me, and I hope that a lot of you are because I find it really exciting to build these content items and publish them. If I start a project, it very quickly turns into like, I was trying to make a shiny application and all of a sudden now I have like three pins and two static documents and a scheduled document and like a graveyard of like old content items that were my prototypes. And I've got like an instrumentation dashboard off to the side that's watching the usage of this shiny application. And this project just explodes.
And now you've got five or six content items that are all part of a single project. You don't have just one report that's the output of your data science work. Now you've got this body of work that all needs to be tracked and understood as a cohesive unit. Sometimes that means like you've got additional collaborators coming on board. There's all of this complexity that comes with building out data science artifacts.
And as far as Connect is concerned, making something that adds a layer of content discovery at this sort of complex project level has not been something we've yet taken on until hopefully now. So I've not been the only one experiencing this issue. We have had lots of other customers reach out to us and I wanted to acknowledge those things we hear you and really appreciate the feedback at this level.
What we've heard from customers is that you're producing a lot of pieces of related content and you wanna share that all with viewers but you're not sure how to do that now given the tools that we've had in Connect historically. Giving your stakeholders pages and blanks is tiresome. The default Connect content dashboard isn't polished enough to share. You want to create sort of an app gallery or a content gallery app store for your content items on Connect. You wanna have custom landing pages for business stakeholders or different groups. You want to make the content dashboard less overwhelming and content discovery easier.
The final piece is a piece of feedback that we took and then changed during the design process. And I wanna give you a little peek behind the curtain at our design process and how we built out this feature set. So the final piece of feedback we got on this was it would be amazing if Connect could automatically generate a gallery. And that was something we really did consider here and we decided to move in a different direction. And I hope that this will still satisfy the core sentiment of this ask in what we built but it's something that we could revisit over time.
At our studio, we really cherish it and love to think of our users as folks who write code. It makes us very nervous to do things automatically for you. There's lots of magic that exists within the package ecosystem. But at the end of the day, by using the R package ecosystem, Python libraries, by writing things out in code you're making those patterns reproducible for yourself at least in theory that's more reproducible for yourself at least.
But at the end of the day, by using the R package ecosystem, Python libraries, by writing things out in code you're making those patterns reproducible for yourself at least in theory that's more reproducible for yourself at least.
So when we designed a solution to make content discovery a little bit easier to make these app stores or content galleries a reality in Connect, we wanted to make that solution as self-service as possible and rely on existing familiar Connect content types rather than produce something that would magically pull together your content items for you and make the end solution more rigid and less reproducible that we wanted. So we wanted it to be code-based. We wanted it to, again, like I said, to be built upon familiar known Connect content types like R Markdown and Shiny. We wanted the end result to be still polished, have presentable defaults out of the box. So not need or require you to do a lot of heavy-handed CSS or work on the design front to get something that looks really great.
We didn't want you to need to read the entire Connect server API documentation book in order to figure this out. We wanted to do some light hand-holding when it comes to working with our APIs and we wanted it to be as customizable as possible and really easy to put your own branding on. So we produced an R package called Connect Widgets that provides HTML widgets or helper functions to organize and curate your RStudio Connect content within an R Markdown document or Shiny application. That's a lot of words, so I'm gonna demo it just to show you sort of what all of this means.
Connect Widgets demo
Let me flip over here to RStudio. Okay, so the Connect Widgets package is available on CRAN. You can install that package's Connect Widgets. And it's already loaded, yay. So I'm not gonna update it, but it does give me great joy just to type out and install that package's Connect Widgets because it was a long time getting it onto CRAN and by a strange manager, Jill is sitting there smiling about. So we're really happy to have it on CRAN and it's awesome to be able to just type that out.
So once you have the package installed and loaded, five minutes, Connect Widgets, you can come over to open a new R Markdown document. That's the easiest way to get started with Connect Widgets even though you can use these components in R Markdown or Shiny applications because if you're using R Markdown, you can start from a template page. So we have a Connect Widgets template available along with the package template. And it will open up into a template right here that you can just knit right away and get started with.
And so the first thing you do is pull out the title and then this will take you through some intro code chunks to get started with Connect Widgets. And you'll note that it does also add Decliner because one of the things that you can do with Connect Widgets is pull all of the content that you have access to on your existing Connect server down locally and then use basic Decliner that you're used to using to filter that down into the subset of content items that you actually want to showcase on this page.
So the setup is to connect to the Connect server that you'd like access to. And the way to do that that we recommend is setting some environment variables. So behind the scenes, I've already set up a .environ file to set my Connect server environment variable and the Connect API key environment variable. So both of those are already set. They're set to that sales server I was demoing before. So once I connect, it's going to try to pull down all the content items and then create a variable called sample content, which will just slice 50 of those content items out for me.
All right, so the first thing we get is not actually part of the package itself. It's sort of a joke that my friend David put into the template. As you can see here, a random unsplash forest image is calling random forest. So we can yell at him about that joke.
But each of the components that make up Connect widgets are expressed here on this template page that you can pull out and push different content items onto. So we're working with this sample content group and the first component type is called a card. So this is just a random piece of content that we've pulled off of the sales demo server that Nick produced on August 17th. He didn't produce, he didn't add any metadata like a description to this content. He just gave it a basic title and then that was it. It's an API type. We can tell that from the default image. And then if we wanted to visit this piece of content, we could click through the open content button. So that's a card that represents singular content items.
You can put multiple cards on a page. You don't need to use the card content type. You can pull it out and you can use something like a grid. So if you want to represent more content in less space, you could choose a grid instead. So this represents a larger number of content. There's pagination. All of these thumbnail images are the default ones that come with the package. If you haven't set thumbnail images for your content, they will appear as these defaults in gray in each of the grid and card types.
Now, this is a special case. I'm not able to pull down the thumbnail images to my local development environment off of the sales server. Most of our sales server content does have thumbnail images associated with it. So you'll see this difference between development and pushing something up to your actual Connect server. If I were to publish this content item back to our sales server, and I'll have that running in the background as I keep talking about the different component types here, but we'll see what it looks like to differentiate between local development. I don't see any of these thumbnail images and pushing it to production where the thumbnail images come into play.
So that's the grid and card. The last content component is a table. So this is a set table of metadata. You get a name, a type, an owner, and last updated, and you get the mini content thumbnail images alongside. Again, it's paginated. And finally, there are components for searching and filtering. So this shows against basically the same table type. If you were to add searching and filtering to this, so I can search for content that Edgar has produced. And the type, let's see, let's look at all, oh, no, we don't have any of, let's see, all of Edgar's pins. It looks like he's produced a lot of pins. And you can also filter by tags. You can look at all Edgar's access to care pins. Searching, similarly, if I were to look at, let's see, maybe that's his model. I just get this content that is named model. So the search just searches the main field here. You can reset, go back. And the searching and filtering component can be applied to the table or to the grid.
Theming and customization
So that's all there is for the basics of Connect Widget and its components. Definitely want to talk about theming, but let's see if I can get this first iteration of it published Connect first before we make some changes here.
Kelly, I can ask a few questions that came in on the Slido as well while it's publishing. One of the upvoted ones is, do we actually push the output of the R Markdown knitted or the R Markdown doc itself? So for example, do I have to set things up to re-knit regularly to keep the list up to date?
Yep, so if you wanted the sample content to regenerate, or if you had this content coming off of a tag, then you would want to publish with the source code, and then you would want to schedule this to re-update. So I could run it, you know, I could run it every 30 minutes to re-update if I wanted to. I don't recommend running every minute. That seems heavy handed. If you're down to that level, then you'd want to probably put these component types into a Shiny application and make them interactive. And that way, at least everyone coming to a fresh session would see the content of it that they should at the beginning of their session.
Okay, so now you can see what this looks like when I publish it. So this is another interesting piece of, I don't know, a piece of the puzzle here is that some of these content items I've pulled down, I don't actually have access to. And that's because on this server, I am an administrator. So I can use Connect Widgets to pull down actually the full set of content items that exist on the server. If you're a publisher, you only get to pull down the content items that you own and that you're a collaborator on. So that's something to keep in mind as you use Connect Widgets.
As an administrator, I need to be sort of more aware of this and more careful because I can produce a content item like this one where even I don't have access to view the thumbnails for each of these content types. So for this one, this is something that I know I own and it has a nice thumbnail image. I can visit it. For this one, let's see. I'm sure that Nick won't mind, but I wanted to show you, this is a feature that's been added in the last six months to Connect where we've put in a request for access workflow.
So as actually an administrator, I don't have the ability to send access requests. I can just add myself to the content, but that action will be audited. So I can add myself as a viewer or a collaborator to the piece of content. If I were logged in here as a publisher, I would see these same options. I could request access as a collaborator or as a viewer, but it would send Nick an email letting him know that I had requested and asking him to give me those permissions. Similarly, for a viewer role, you only have the ability to request view access since you don't have publisher rights and it doesn't make sense for you to be a collaborator.
But now you can see that we still have a couple content items that have the default image. We have a couple of content items that have been updated to their existing thumbnails. And this is a table that looks the same. So the table and the search and filter table, they look the same between development and their production state on Connect. But you'll see those differences for all of the content types, card and grid, that allow you to associate a thumbnail image on something.
So that is Connect widgets. I did wanna show the theming elements too, since this document that I produced is pretty boring at this point. Let's quickly add a theme here. So I'm warning you, I'm not a great designer, but let's see, we can use the Bootswatch themes, which are really nice. And then often what I'll do in addition to this is add a background and foreground color as well, so that some of the colors can like fill out the background and foreground. I find that just adding the Bootswatch alone doesn't always like satisfy my need for enough green on the screen.
And that's what you'll see I've done for some of the images that I mocked up for the blog post and for the logo that Rachel used to put together. But this will show you just what you can do, how these things translate into different component items when you just add a Bootswatch theme. Beyond that, you can really go to town. If you want, you can add as much CSS as you want to this, but this just shows you what Minty will do. So it changes the font, it changes the color of the buttons, it changes the color of the highlight text a little bit, and the pagination. Again, all the Bootswatch themes will take you in different directions, but translate into component items themselves. So you can turn up the knobs on that, like I said, by adding background and foreground colors to add more different accent colors to your content.
Filtering content and building a curated showcase
So that is the theme. And the other thing that I wanted to show was how to take the sample content and turn it into something that might be actually useful to you. I'm assuming that most folks on the call, if you are at all interested in this feature set, are not going to want to pull down a random sample of content off of your server and then put that onto a page. So if you're interested in interacting with Connect Widgets to pull down content that's meaningful to you, there are two helper functions you're provided that you should probably know about. One is by tag.
So by tag, let's see. I have one picked out for Python. Is it just called Python? Let's look. So this is all the Python content we have. Yeah, so it's just called Python. So let's see what happens when I pull down everything with the Python tag. All right, so now sample content has 47. We've got 47 different pieces of Python content here. And that's a lot, so I'm gonna filter that down even further by, what's my name on the server? I think it's just Kelly. Kelly. All right, so now I'm down to nine. I've only contributed nine pieces of Python content to the sales. That's okay. We'll use that.
So now this will take the same sample content variable, but now it contains Python content that I've produced, and it'll put that into the card grading table. So let's make that, let's see, what was it? Okay, so here's all my Python content. That's great. I don't need the table twice. I probably don't need the table at all. So I'll just keep the grid and the card, and then change this to Kelly, Python.
And in the meantime, I'll talk through the rest of my slides. So it gives you a bit of a sense for what Connect Widgets is capable of, I hope. It gives you a sense for what you can build with Connect Widgets. In the backend, remember that you need to provide an API key and server adjusts environment variables because it is using the RStudio Connect server API to do those calls out to your server to pull the content items down and give you that information about the metadata associated with each of them. And then you can apply different groups of content to different HTML components and place them on a page or an application.
And I'm hoping that this is useful to folks, not only for the case that I described at the top of the deck that talks about like a project that just blows up in complexity, but also as potentially a content hub or knowledge repository for a whole group of sort of related content items that belong to a single group or an objective or an OKR or a team as a single entry point for stakeholders. So if you wanted to create a really polished showcase for a stakeholder group that just shows them the content items that you want them to see and doesn't point them to the overwhelming leadership information that is the Connect dashboard, you could use that for this, or just as a general purpose presentation layer for any curated list of content items.
So if you wanted to create a really polished showcase for a stakeholder group that just shows them the content items that you want them to see and doesn't point them to the overwhelming leadership information that is the Connect dashboard, you could use that for this, or just as a general purpose presentation layer for any curated list of content items.
This is another interpretation of something you can do with Connect, which is for a more complex project. I'm sure some of you have seen this demo to you by our solutions engineers or sales team, but this is an end-to-end data science workflow for visualizing and predicting pipe share data.
Oh, oh, the thing that I published is now ready. Here's my Python demo content. Oh, it looks great, very cool, very cute. Okay, sorry about that brief interruption.
So the actual demo assets that build up this set of analyses for the pipe share data is built upon a number of our Markdown documents that do data import, cleaning, and model training, and produce output in the form of several PIMS data sets that get updated on a regular basis. Those then get pulled into API prediction services built on Parmer and an application for doing an assessment of model quality over time. And then the final output is two different prod and development client apps that can be rotated between each other.
So all of these components make sort of this ecosystem that you can move on to this complex data science project. And you can use Connect Widgets like we've done here to sort out all of these different content items and make sure that everybody who's collaborating on them knows where to find them, knows what all the pieces are, knows that when something breaks, you can refer to this diagram and see what else downstream might be affected. And use this as not only a landing spot for everything that's been deployed to Connect for this particular project, but also link back to the GitHub repository itself where we manage all the code that goes into producing all of this content. So everything is linked together. Everything can be shared as a singular showcase. And we're hopefully reducing the amount of complexity that there is to track over time.
Supporting features: request for access and Git-backed publishing
As I mentioned, there are a couple of supporting features that I demoed that went into some early releases of Connect that have built into this 1.9.0 release. One was the request for access permissions that I showed. Another is more streamlined publishing. So even though you still have to set the Connect server and Connect API key on the local side, local development, these will be, by default at least, automatically enabled for content that you publish to Connect. That's why you'd see me be able to publish directly these things that I've been building locally and they are just working on Connect. I don't have to go in and set my API key or server address again, once that content item is published. So it doesn't publish in a broken state anymore. That was very nice.
If you are a particularly security-conscious customer, your server admin does have the ability to turn this particular feature off, as well as the request for access permissions. So those are always options for you. If these are offensive to you in some way, the product will continue to work as it did before.
I see there was a question where someone asked, is there a way to pick up changes from Git when you make new commits to some of the included content or how does updating one piece of content work this way?
Yeah, absolutely. So if I go into Connect, I could, let's see. Well, yeah, let's change this. I think we have time. Let's do this quickly. So I'm going to create a new GitHub repository in my own personal GitHub for this content. Hopefully you have figured that out internally.
So in this step, I won't show where I'm creating a repository in my GitHub. You'll do that for your own organizational repository. Okay, so now I have a new repository. The last piece that I need to do is to generate a manifest for this content item. So I will do, let's see, rsConnect.
Right, manifest, and now I'm not in the right content. Let's see, set as working directory. Now I'm at the manifest, so that it goes into the template example directory. And then once that's written, it takes a couple of seconds, but it's trying to capture all the information about the packages that I used and the version of R that I'm running and the content itself, including this banner image. And it's writing up a manifest of all of that information for me so that I can commit it to a GitHub repository and then I can link that Git repository to a content item on Connect. I will have to produce a new content item. I can't link the existing one that I already push-button published to Git. I have to start over from a fresh content item. But since I've only published once so far, it shouldn't be an issue for me.
Is there an upcoming feature so that publishers can create tags? That's a great question. I'm so glad you asked. We don't have that slated for an upcoming release, but it is something that we have heard a lot. So I am quite aware that it's a hole that's necessary to complete this publisher enable
