Posit Cloud Essentials | Ep 3: Speed up your data workflows with Project Templates
On the last Tuesday of every month, we host an event – Posit Cloud Essentials – where we explore the ins and outs of Posit Cloud, diving into its key features, valuable tips, and real-world use cases. The event is open to all and hosted on YouTube with a live Q&A during each month's event. THIS MONTH'S EVENT: All data projects are not created equal. Sometimes you need certain packages. Sometimes you need specific data files or helper scripts. And sometimes you want to define specific compute resources to match your analysis. Creating a new project in Posit Cloud provides you with a blank canvas. Project Templates enable you to add structure to your canvas, take a snapshot of the newly defined environment, and reuse that starting point for subsequent work. This episode of Posit Cloud Essentials will showcase how you can get started more quickly on Posit Cloud with Project Templates, either by using one of our predefined workflows or building your own. WHAT IS POSIT CLOUD? Posit Cloud makes it easy to move your entire workflow into a unified online experience with project management and publishing capabilities. Use your favorite coding languages and environments and share your work seamlessly with others, all from the comfort of your web browser. ADD THE EVENT TO YOUR CALENDAR: No registration is required to join the events. Simply add the event to your calendar using the link below. Add future Posit Cloud Essential events to your calendar → http://evt.to/adahaeuow EXPLORE MORE ABOUT POSIT CLOUD: Create a free Posit Cloud account → https://posit.cloud/plans?utm_source=youtube&utm_medium=organic_social&utm_campaign=cloud_launch View the entire library of Posit Cloud Essential events → https://youtube.com/playlist?list=PL9HYL-VRX0oS4CXCA81eno41u8K3ckGVH&feature=shared
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hello again, everybody. My name is Alex Chisholm. I'm on the Posit Cloud team here at Posit. Thank you for joining us for this third episode in our live demo series. Today we're going to turn to a new feature that we put into the product in the middle or late August called Project Templates.
And what Project Templates enable you to do is speed up your workflows when you have a core set of items that you want to put in the same project over and over and over again. We make it easy to surface those, and we're going to take a look at that today.
So you probably are aware, for those of you who have joined before, once you're on Posit Cloud, it's pretty easy to start a new project. Typically, you're going in and you're just clicking New Project and you're creating either a Blank RStudio project or a Jupyter Notebook project. Maybe you're pulling something in from GitHub, from a GitHub repo. And what we've added to this flow is this ability to create a new project from a template.
There are several circumstances, right, where this functionality might be useful to you, because every time on Posit Cloud you're creating a new project, it's really a blank canvas, a blank slate. It's up to you to decide which packages to install, which data connections to make. And what you can do with a template, if you find yourself with a recurring workflow, maybe you always use the same set of packages. Maybe you need to bring in data sets or flat files that don't change very often. You want them to be used across all of your projects.
Maybe you know that a certain type of project is going to have a lot more compute requirements and you want to be able to set your memory or your CPU at a higher level. Or maybe you just want to get into some of those fine details that you might find in something like project settings in the RStudio IDE and you want this to come up every single time that you're opening this new project. So this is what project templates enable you to do.
And there are two ways for you to start interacting with templates on Posit Cloud today. The first one is you have the ability to select from, I think, six predefined, pre-configured templates that have been put together by thought leaders and experts around Posit and that are now on Posit Cloud. So that's the first way. It helps you get started more quickly. You can see some examples, see some sample code, data sets, and have all the libraries and packages you need to work on those workflows without setting up stuff for yourself.
Secondly, you have the ability to design your own template. And this is the one that's going to become more powerful the more you use it, I believe. So here you're going to be able to create a new project, do everything you need to do based upon those four items we mentioned on the previous slide. And then you're going to be able to call this template again and again and again when you're doing subsequent work. And we're also going to see the ability to make what you do in this template to default anytime you start a new project from within that space.
Live demo: exploring Posit Cloud
So as we're going along here, we do have Q&A open up. I'll probably jump in at the end and answer anything specific that comes in if I don't see it while we're going through it. But right now I'm going to hop over to a free account on Posit.Cloud.
OK, you should be able to see my screen now. There it is. And I'm going to hop off and we're just going to go into Posit.Cloud. For those of you who might be new, who need help getting started, all you have to do is click on Get Started. You'll see a variety of ways that you can engage with Posit.Cloud. Again, I'm going to go ahead and use a free account today, which I've already created on this machine.
So if I log in, I'm just going to go out to Google here. It's going to authenticate and bring me into the product. And the first thing you're always going to find, right, is this Your Workspace. And from Your Workspace, you see the button that I looked at on the previous slide talking about what that new project would be. And you can see that now we have this additional new project from template.
So if this is your first time, if you click on New RStudio Project or New Jupyter Project, you're pretty much getting just the basic components of what you would have without any additional packages installed, really just what you need to get started and get running. If you click on New Project from a Git repo, you're going to be able to go ahead and bring something in from GitHub, which would automatically create that RStudio project for you. And what we're going to look at today is New Project from Template.
Using Posit Cloud's built-in templates
So if I click on this, you're going to have a lot of options come up on your screen. And you can see that we're saying that you want to create a new project from a template. So here's a list of available templates within your space right now. And these are all put together by us from Posit Cloud. So maybe you're interested in doing some machine learning with tidy models, or you want to do some publishing maybe with Quarto, or you want to make an application with Shiny app. All of these are going to bring in all of the libraries associated with some of the sample code that we have in here for you.
And I know many people on the call are educators. And when you're working with students maybe before the semester and you want them to get started, it's kind of hard for them to create a new project because they don't know how to interact with R or Jupyter. From clicking on one of these, they're going to get set up in a way where they're going to have something to play around with and click with immediately.
So let's go ahead and click on Data Analysis in R with the tidyverse. I'm going to click OK. And it's going to create a new project for me. And let's just call this tidyverse.
It takes around 15 to 20 seconds to create a new instance for this specific RStudio project. And because we created one through a template here, we're going to see a variety of files in here when we first get started. So this sample template, we'll notice a few things when we go over to the Files pane. The first is that we have a data folder. All right. So we've got some data that's already in here in CSV format. We also have a QMD project and a readme.qmd. All of the templates are going to come with this readme file. And this is going to give you a little bit of background about what you can expect from the code and packages that happen to be in here.
So in this case, we're going to go out and explore some languages spoken in the United States. You're going to be able to run some code in the specific file languages.qmd. Tell us a little bit about the data that is available in that folder. And all the files are in there. And then which packages have been included already from within this template so you can get to work right away without having to install all of these.
So let's click on that file, languages.qmd. All right. So we've got a Quarto document. That's what the QMD is. For those people who have used R Markdown in the past, this is kind of the next generation of that tooling that we're really pleased with at Posit. I'm going to go ahead and render this in my viewer pane so that we all can see it relatively easily. And there's not a pop-up that might come out from the window that I'm sharing here.
So if I go ahead and render this, once again, without bringing in any additional libraries or packages, we're going to be able to see, you know, the analysis that came from this workflow. We went out. We read in the path for where our data is. We started to do some exploration, see what the main variables look like and the top records are. Group things together. So we're looking at all the languages. We're trying to find out what is the most common language spoken aside from English. And then we get to the point where we're doing some visualization, right? So Spanish, Portuguese, and Chinese is coming from this data set. You get the idea. We're just doing basic exploratory analysis here based upon a predefined set of data and packages associated with this.
Creating your own custom template
So this is using something provided by Posit Cloud. If I go back to your workspace, you're going to see that now what was the template is saved as a piece of content, a project. I can play with these settings a little bit. So I already named it the Tinyverse demo when I went in. Maybe I don't want this description to change anymore. Maybe I just want to do, you know, language analysis. We can customize it. We can decide who gets to see it and who doesn't. Right now I'll just leave this private.
But you can see that we took a predefined template created by Posit Cloud and we let you, the user, save this as a new project from which you have all those assets that came from it. And now this is just a regular project as you would have seen before.
And what I want to do here, you'll notice one thing when I click on New Project and New Project from Template. It's only from Posit Cloud. What we have the ability to do within a shared space is have you define a template of your own and then have you and others that maybe are members of your space come in and start using those customized designed templates. So let's create a new space. And we will call this Shared Space maybe.
We've created a new space. As we've talked about in the last live demo where we did a deep dive on what it means to be within a space versus a personal workspace. Two of the major differences at the top here at least were the ability to store data connections and to have member management. So maybe you want to bring in a few more people into your space that you're collaborating with. That is fine. But what we're going to do is take a look at the template flow here, which is slightly different.
So if I click on New Project and do New Project from Template. Now you're going to see the same set of six templates from Posit Cloud that you could use in the same way that you were doing it in your personal workspace. But you also can see, well, you could have some templates coming in from this specific workspace that we named Shared Space. But there are no templates available at the moment. So let's go ahead and create one so that we're able to come back here and pull it up.
So to do this, the first thing we're going to do is create a new project. And let's call this Marketing Analysis. Maybe we have new data that comes in every month. We want to be able to run some analysis on it. We've got a core set of data that we're going to append to here. We've got some libraries that we use.
So when you click on Packages, just as an expiration, my guess is that many of you on the call are used to using the Tidyverse in a lot of your workflows. Because of the way Posit Cloud works, every time we create a new project, you're getting essentially this blank slate, this blank canvas. So something like Tidyverse won't be there. So what we can do is to install Packages. I'm just going to bring in the Tidyverse. It will take about a minute to pull together everything. Because, of course, we're getting ggplot2 and we're getting dplyr and a bunch of other helper functions that are part of that Tidyverse. So we're going to go out, and this is going to expand the packages that we have in our space.
Maybe we do have a data file that we're going to want in all of our subsequent analysis within the space. So maybe I can upload a data file here. I'll just go out and grab a quick one that I've got on my desktop called Marketing. And we can see that now we've got a marketing CSV file that is quite small.
So, so far, we've added a package, the Tidyverse in this case. If I go back to Packages, now I'll be able to see that Tidyverse is available for us to load in. And our list of packages has expanded widely. Something else we might want to do, maybe we want a recurring script. So maybe we want just a very basic pda.r, where I'm going to go out. Oh, the Tidyverse, maybe. There, now we've got that. And take a glimpse or a look at our data set.
So I'm going to go to my data folder, and I think I named it Marketing.csv. Oh, sorry. First, we're going to read this in. Read CSV. I'll just name this my data frame. We brought this in. Now I'm going to go take a look at that data frame that I brought in. So now we can see the types of variables that are in this relatively small data set, but it does have 39 columns, so it's fairly large. And we can see that we have the income of a specific person, how many kids are in the home, how many teenagers are in the home. And maybe this is something that is part of a constant flow that we're always pulling in as sort of a starting point within our analysis.
So now we've brought in packages. We've brought in data. We've created a script. The other thing that we could do through templates is change our compute resources. So one thing that is unique about Posit Cloud is our ability at the project and output level to determine how much memory we want to use and how much compute we want to use. And then finally, what the background execution time is, meaning if you're not interacting with your console, how long will the project keep going to work on some of these background tabs? I'm sitting in a free account here, so it's going to tell me I can't go above one CPU or one gigabyte of RAM. But if you were on a basic or a standard or any other paid plan, you'll have the ability to increase these, save them, and then come back to this as a template with having everything kind of put together in the current form.
Saving a project as a template
So now that we've gone through and brought in all of these new components, let's go back to our space. We see that we still just have this private marketing analysis project that we created. But let's turn this into a template. If I click on dot dot dot and click Settings, again, I could add a description, I could change the name. But if I go over to Access, I could keep it private for myself, like if I'm the only person in this space and I wanted to use this template myself, I could. If I do bring in new space members and I want them to be able to interact and use the template as well, I need to make it available for everybody. And then we can look at special access here. And there's a new checkbox called make this project a template.
So let's start with this. I'll close this. The first thing that we see is that we have a template tag that is now on the project. It's going to fall within that template folder so I can know how many are within my space. And now the fun part, the useful part, is when you or anyone else that you've invited to be a member of your space creates a new project from template. Now we have this marketing analysis template there for us to create a new project.
So if I go ahead and click OK. And this could be, you know, now we're going to do our October 2023 reporting. But because all those things that we did the first iteration, we added packages, we added a data set, we added a script, we could have changed our compute resources. I don't want to have to do that every single time that I'm operating here. This should bring in the right files that we added. So we can see that we have EDA.R. We can see that we have that baseline marketing data set that we use as a starting point for a lot of our analysis. And when I go ahead and try to run library tidyverse, it's going to go ahead and load because it's already been configured with everything that we included within that template.
And when I go ahead and try to run library tidyverse, it's going to go ahead and load because it's already been configured with everything that we included within that template.
So now in terms of all the content we have, we just have one derived project from that template. And we have the template that we had the project that we marked as a template, which then continues to serve as a starting point for other people who want to start something new.
Setting a default template
The final thing we can talk through here, before getting to some Q&A about the new feature of Posit Cloud in general, is if I were to create a new project, a new RStudio project right now, or a new Jupyter project right now, both would continue to be blank. So templates give you the ability to have a specific one based on certain workflows to help you get started faster and then reuse them when you find yourself with recurring or repetitive work.
What you're also able to do, if you know that you're always going to use, let's say the tidyverse, we can also take the template over here, marketing analysis, and go back to the dot dot dot. When I click on access, I can see that this already is the template. It's the one that we use to create the subsequent project. I can also go one step further and say make this the default RStudio template. Meaning every time that anybody within your space creates a new project, they're always going to get whatever was included within this project.
So now if I come back here and I click on new project, this and every other new one that you would be able to create, you should have those same assets that we looked at before. So created from default templates. Once we get verification that we have the assets coming through from here, we sort of covered the three different ways that we can use templating to get up and running more quickly on Posit Cloud. And there you see we've got the scripts, we've got the data files, and the library should be there as well.
Q&A
Awesome. Thanks, Alex. We had a couple of questions come in throughout the presentation, so I'm just going to start going through them. I think Jeff asked a clarifying question. He said, so every time I make a project in Posit Cloud, I have to download and install packages?
Yeah, that's the case. And the way that we set this up right is we want each and every space and each and every project to be kind of a standalone entity where you have total control over what's in there. And that includes packages. So what this feature enables you to do from a variety of ways is say, well, I don't want to download something every time if I know I'm going to use it every time. So we can either create a template for that, or we can turn that into a default template where you don't even need to think about starting a new project flow through that templating functionality. You're just getting those packages off the bat without having to install again and again.
James asked, are we able to connect to Azure SQL?
Yes, you should be able to. Let's go. Let's go over here. So in essence, anything that you can connect to via the RStudio IDE, you should be able to do it through Posit Cloud. The one thing that we added specifically in terms of data connections more recently, and I think this came out in May, was the ability for Posit Cloud to securely store your credentials for certain data connections for which Posit maintains a set of professional drivers to help make this connection easier for you. So I don't think Azure ML is going to be here directly, but you can see, I think, 16 of the most common database connections that are out there. And if I click on Postgres, I can save all of my credentials, and then I can bring this in through the project. So even though Azure ML is not here, when you're within your project, if you can find a way through RStudio, through a package or another driver that might be included to make that connection, that flow would also work through Posit Cloud.
No. Yeah, right now, this is all Posit Cloud-specific in terms of functionality. So both the data connections and the templating are proprietary in some ways to Posit Cloud at the moment. And what we're trying to do is we're trying to add some bells and whistles around the baseline functionality of the standalone IDEs in a way that carries through throughout the product. So you're able to go between a template or a data connection and these projects and outputs in the same space.
Will templates for Jupyter projects be available in the future?
We've definitely been considering this and we've had on our roadmaps some Python work for a while. You've probably noticed when I'm going here to new project from template, we do have a templating here. So let's go ahead and click this. But I'm not in a free account. Let me switch over to my other screen for a moment.
Yeah, so now I'm in my Posit account. I'm in my personal workspace. I'm going to go ahead and create that pandas template. So we have the ability to create this from the global templates put together by Posit. So this is the Jupyter template. And this was worked on by Dan within our company, going out and putting together some sample data files and the ability to go through and use pandas to do some basic data analysis. So once again, here, we're able to save the penguins clean. We're able to find a penguins clean CSV. We can go into EDA and without downloading or installing any of this, let's run it to make sure that it works. But I've gone through and I've loaded in the required libraries and I can come through right away and see I'm able to connect to that data set and put together a table like this. Let's go out to your initial question. Let's go out and create a new shared space.
Let's see. Jupyter Template testing. I'm going to create this. Let's make a new project through Jupyter. Because I think through this now, I'm thinking this actually will work in terms of setting up and saving a project from Jupyter as well. That's interesting. I think if I just refresh, it'll come up. There we go. So test template. And let's go ahead here and maybe do a new notebook. We'll use Python 3.10
Just to test this here, I'm just going to name this, you know, load pandas. Go back to make sure that it is saved. Go back here and we see that we have load pandas here, even though we haven't brought in any specific packages at the moment. So let me go back now to our space. I'll go ahead and make it available to anybody who happens to be within my space. I can also make it the Jupyter default. So it looks like we do have full functionality here in terms of the Jupyter side for for templating. And then we can validate this by now going back to new project, creating new project from template, and we'll find that within this space.
So for those of people who do have a paid plan or working with, like in a university setting with professors who have a paid plan, you should be able to also create Jupyter templates, both from the global Posit Cloud templates, as well as setting their setting their own. Just wait for this to come up to verify completely. But I think it should work. And here we go. This was the template that we created, where we just went out, we made a quick notebook, and we brought in pandas. So it does, it looks like it is working.
Yeah, totally. So I looked at the data connection feature with you a few moments ago. We're just going through and we're making this connection. If you were going to be able to use those within one of the templates that you're defining, as long as that initial data connection still exists within that space, they'll be able to carry through and be used through templates.
Awesome. Then Cedric asked, I set up the use of observe event as a template. This template works in my own RStudio workshop environment, but it fails to react in the Posit Cloud environment. So I think the question is, any ideas on why that might be?
So an observe event within Shiny, I'm assuming. This one's a little bit harder to tie notes off the spot. But feel free to email me at alex.chism at posit.co. I'd be glad to work through with that and find out what might be happening there.
Perfect. So right now, that wraps up all the questions that we got for today. So unless any come in last minute, we'll wrap up. But thank you all for joining. Thank you, Alex, for showing us how to use templates. And just a reminder that we do these events every month on the last Tuesday of the month. So you can check back in and we'll be covering another Posit Cloud feature or use case in October. So thank you all for joining. Thank you, Alex, and see you next time. Sounds good. Thanks, everyone.