Launch different development environments and manage cluster options with Posit Workbench
Posit Workbench: https://posit.co/products/enterprise/workbench/ Data scientists should be able to use the language and development environment they prefer. Jupyter Notebook, JupyterLab, VS Code, and RStudio are all available development environments within Posit Workbench. Workbench is also exceptional for managing compute resources. Use Kubernetes and Slurm and adjust the CPU and memory to match the job you're trying to run
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
And this again is one of the best parts about Posit Workbench, is that no matter what your data science team uses, their preferred IDE, the hope is that we can support it here within Posit Workbench. So we currently support Jupyter Notebook, JupyterLab, we have the RStudio IDE as well, and then we have the VS Code Editor, which is a great all-purpose editor, which is what we're going to use today. So I'm going to click on VS Code, and when we do that we get some additional options. So I can give this VS Code session a name, I'll just leave it as a default right here, and then right underneath you're going to see Cluster Kubernetes. Now to tell you a little bit more about our demo environment here at Posit, it is all hosted in AWS, Amazon Web Services, and it's in a high performance computing cluster, an HPC environment, and that environment is managed using something known as Kubernetes. So some of you may have heard of Kubernetes, that's one of these HPC managers that we support. If you also use Slurm, we can support that as well. Or you can also just run these tools on a single server, that's totally fine as well.
Managing compute resources
But one of the advantages of using Kubernetes or Slurm is you can actually control this job's CPU and memory allocations. So if you just have like a small job, you want to maybe create a plot or a simple application, you can keep the CPU low, keep the memory low, and that's just great for managing compute resources. But if you have a higher compute, a more demanding job, I can bump that up, for example, I can change the memory as well. So really good at managing resources.
But one of the advantages of using Kubernetes or Slurm is you can actually control this job's CPU and memory allocations.
Then you'll see this last box right here, and this is the image box. So I have the option to choose a specific Docker image here to deploy this session. So that can help control additional environment variables, things like what R versions are available, Python versions, what packages are installed by default, that can all be controlled using various Docker images. But I'll just stick with the default for right now. That all looks good. Let's go ahead and start the session.
