Using controllers to write robust Shiny for Python app tests | Karan Gathani

Transcript#

This transcript was generated automatically and may contain errors.

Hi everyone, I'm Karan. I'm the QA engineer in the Shiny team and today for this video we'll go over how to leverage some of the methods that we exposed for testing your Shiny app and with the release of version 1.0 we exposed a bunch of helpful functions that you can actually use to actually interact with your components in your Shiny app and then verify they work as expected.

This exercise I'm going to use so I already have two tests created so I have one which is so this is the app file this is a basic Shiny app so let me show you what this app does so it has a slider and then there's a text output so if you move around the slider you'll see the text output updating and it essentially just doubles the value so if I have it as two it'll double it before so so by default when the Shiny app loads it's always at 20 so the output text when the thing loads it's always going to be n times 2 is funny and so for the test what we'll do is we'll use some of the methods we exposed that make will make it much easier for folks to write their own tests we'll probably move the slider to 50 and then we'll verify that this thing updates so let me go back and retread while we load so we'll load the app we'll check if this is by default this is the version or this is the value that when the slider value is at 20 we'll interact with this move it to 50 and then verify this thing is at 100.

Okay so that's the Shiny app and this is just a basic Shiny app that I used using the Shiny create and then selected the basic app so this is the express version of the app in case people are curious.

Using controllers in tests

All right so for the test what we will do and I'm going to close the Shiny app since we don't need it. Okay cool so what we do in these lines of code is the app when you use the Shiny test add utility function like the command line it will automatically fill your test file with page go to which will actually navigate to the url of the Shiny app and since the Shiny app is running locally it will have a unique port and all the fixtures over here will automatically determine that for you and so you don't need to worry about that.

So over here so we have two components when we look at the app file so we have an output text and we have input slider and we want to test both of them so for this one what we do is we use something called as a controller which allows you to interact with your different components in your apps so what I do is I do controller dot output text I'm creating an instance of them and then you end up passing the page object and this is the id of the so if you look over here you can see the id of the output text sorry over here and then what we'll do is oh sorry not this one okay cool so we create an instance of that with the output text file we'll create just another variable and then we actually expose some methods so if you're using vs code you can actually figure it out and then this one when you actually before you fill it up it'll show you some information of what it accepts so you can always do something like this n times two is 40 which is the default one so this is the basic of the first one that we had wherein when you load it it will always show you what the default one is but now we want to actually create an instance of the slider as well so I'll do slider equals to controller so controller you'll always have to invoke it from the controller because that's the one that allows you and when you actually create like controller dot you'll actually see all the different components that you can actually create it so what I'll do is I'll do input slider and then again the same thing is the first one you have to pass the page object which is like a right thing and then you'll pass in the id of that and the id of this one is n so if you're ever confused you can always look at the function definition and so that's the id so I'll do n all right so and then since I created the slider I can actually use slider dot and then you can see all the different things and I can always I can actually do something like set to actually move it and then it you can see the value it accepts it's a string it's not a number it's a string so what I'll do is I'll move it to 50 so it'll it should drag it to 50 and then what I'll do is I'll actually copy this and then I can actually do this 200 so the theory is you move the slider to 50 and then this one moves 200 so so what I'll do is I'll run the tests now

and it should end up picking it up and move the slider and verifying it and I also make it fail deliberately to just see if it's working as expected so pass the first test and then the second one so this is the one that we are passing okay so the first one failed because we still have this error so I'm gonna correct it to party so now we should have both of them passing take a while because it's actually opening up the browser and all of this is happening headlessly which means you won't see the browser but yeah so it actually ended up doing it so you saw the failure when we had that this one is just the test file that if you want you can actually look at the other video that we created which was just about getting started with pytest and we created this file in that one but for this example on this video we are actually doing this so you can actually see the different methods that we expose and like I said you can actually do some assertions and it has expect value as well so you could actually do some expect value of so this one would just be the slider so which number did you drag it to so let me see if this thing works and let me actually see so it's a value pattern or store and then this one asserts that the input element has the expected value so if I run this again it should actually expect that so we are asserting that the output text is reflected with the change we also asserting that the slider value so when you actually move the slider which I can show you over here this value is showing up as 50 so that's what we do it over here so all the methods that we expose for different output input elements like the different controllers that you can use to tweak around and use it for your test are all documented in the testing API documentation page in the reference you can actually see all the methods we expose so you can always use that it highlights what function params it accepts.

Waiting and avoiding flaky tests

And like so one of the things that we do and that's something that we do behind the scenes that the user does not need to worry about is we will explicitly do the waiting and we'll keep it like really dynamic wherein if the app hasn't loaded yet we'll actually keep on checking for a while till it reaches the time out and we don't want the test to fail just because it did not load in a split second so we'll actually take some time to actually constantly check if the value hasn't changed and so if we ended up actually failing the test deliberately or if your test ends up failing you'll actually see in the error logs in the stack trace that it ends up actually testing it several times because it's hoping for it to load and this is also one of the benefits you can get by using Playwright because you're not explicitly like sleeping till things load up and we have a bunch of things that we have added behind the scenes so you can see how it's actually trying it several times to make sure it loads so if in case if it was the case wherein things weren't loading right away we would like wait for it we'll give it like a like a timeout like five seconds to actually make sure it loaded before we end up failing the test because we do not want flaky tests.

like a timeout like five seconds to actually make sure it loaded before we end up failing the test because we do not want flaky tests.

All right so this one actually sums up just a basic test on how you can leverage some of the controllers that we expose in the Shiny for Python version 1.0 package that you can use to actually now write tests for all your Shiny apps and make sure these ones as you keep on adding more functionality to your apps or you don't end up causing regression so I think it provides a really high value of making sure users have a good experience with your Shiny app and also they don't actually break the experience or cause any regressions if you keep on if you have multiple people working on your Shiny app and stuff like that so this one makes it a lot more stable.

So yeah so that's the end of video two. If you need more information feel free to look the testing API reference doc that should be highlighted on the video and have fun testing.

Using controllers to write robust Shiny for Python app tests | Karan Gathani | Posit

Transcript#

Using controllers in tests

Waiting and avoiding flaky tests

Featured software#

Shiny for Python

Shiny