Modular, layout-as-code approach for customizable Shiny dashboards (Kim Schouten, De NZa)
Modular, layout-as-code approach for customizable Shiny dashboards Speaker(s): Kim Schouten Abstract: To improve government supervision in the healthcare space, we have designed a Shiny dashboard that enables us to adhere to GDPR and other regulations and share different bits of information with different types of stakeholders. For that, we have implemented a ‘layout-as-code’ system where each user can have its own layout of pages and modules, stored as json in the database. Users have the option to drag and drop modules to different locations or change the layout altogether, adding and removing modules as needed! For developers, this system of loosely coupled modules helps to quickly contribute to the project as a new module needs only a couple of lines of code to integrate it within the framework. Materials - https://github.com/KSchouten/elemental posit::conf(2025) Subscribe to posit::conf updates: https://posit.co/about/subscription-management/
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
So I was saying, my name is Kim, for the people in the stream who missed that, I'm a data scientist for the Dutch Healthcare Authority or NZA. We are a regulator in the healthcare domain and today I'm going to talk a little bit about how we made our healthcare regulation a bit more shiny by making a dashboard which is modular and has layout-as-code so we can accommodate different types of users and be compliant with privacy law and be developer friendly as well. Lots of promises.
So first, if you think about healthcare regulation you need to think a bit about healthcare systems and they have a tendency to be very differently for different countries but we don't have the time to dive into that too deeply. So let's simplify things a lot and have a look at conflicting ideals. So basically any healthcare system boils down to these three ideals. So we want basically good and cheap healthcare whenever we need it, right? So we have the affordability, availability and quality of healthcare. Unfortunately it's actually quite impossible to have all three of these ideals. So there's always a trade-off between them.
And at the Dutch Healthcare Authority we are often looking at this impossible triangle through a more economic lens. So as you know healthcare often boils down to money. It does. So things that we are on the lookout for are things like fraud or misuse of healthcare funds, unexpected bankruptcies, things like that. And we use data and dashboarding to monitor this and intervene where necessary. But to reintroduce a little bit of complexity because this really is too simple, we are not the only regulator in the Netherlands that deals with healthcare. So the landscape looks more like this. So we are somewhere in there and actually there are more than this because some of the regulating tasks are on the municipal level. So basically any municipality has regulating tasks.
And as you can see all of these regulators are basically optimizing for the same problem, right? Again the good and cheap healthcare whenever we need it. Yeah, so same problem. Mostly the same data and mostly the same needs in terms of dashboarding, which is why we decided at some point to build a shiny dashboard. We are in the shiny session after all. And share that with other regulators. So the three organizations over there came together and decided to build a shiny dashboard. And then the idea was to share that with as many other regulators as possible.
That worked out pretty nicely until this happened. So turned out we were sharing data with regulators that we were not allowed to share. Because all of these regulators have different laws, different jurisdictions, making things really, really complicated. And what we were doing was technically not allowed. Yeah, so we weren't allowed to continue with that. So we had to decide either build a different dashboard for each different type of user, which is horrible, or yeah, kick people off the platform, which is also horrible. But that's what we did. So we kicked a lot of users out and only the ones that we were actually allowed to share the whole dashboard with stayed on. But yeah, that kind of defeated the purpose of the dashboard because we wanted to share healthcare information with all the other regulators, right?
The modular design approach
So we had to go back to the drawing board and come up with something better, which is where the modular design comes in. So we thought, well, if we cannot share the whole dashboard or the dashboard as a whole, why not just cut it up into pieces? And then for each of the pieces, or let's say modules, we can decide with whom we can share that. So that should give us enough control to be compliant with the privacy law. So it would look something like this. You have groups of modules and groups of users based on the sensitivity of the data and based on the different laws and who gets access to what. And then it's a matter of just mapping these two groups. So managing user access and GDPR compliancy was the main reason to go with a modular setup.
The second has to do with spaghetti and waffles. Yeah, that's unexpected. That's to get your attention back. Very good. Spaghetti and waffles. We'll get there. First, I'd like to see a show of hands. Who knows what Shiny modules are? Oh, pretty good. Who actually uses them? Not bad. All right. Nice.
So as you all know, most likely a Shiny app has a user interface and a server function. And as your app becomes larger, especially that server function has a tendency to grow and grow and grow and becomes like thousands of lines long and very hard to manage and debug. And the awesome mechanism of reactivity can grow a bit out of hand so that if you change something over there, something over there might break. And that's completely unexpected and it can become a bit of a nightmare. The nightmare being like spaghetti. So that's how your code kind of looks like. You tug on one piece and on the other hand of the plate, things might break. And our code was a bit like that. Our initial dashboard that we weren't allowed to share anymore. It kind of looked internally like this. So when we sort of were forced to do a redesign anyway, well, let's fix that as well. So that's the secondary reason.
And what we wanted would look more like that. Like a waffle. A nicely baked waffle. It's super structured. It has these nice squares and you know exactly what goes where. It's awesome. Yeah. So Shiny modules are a bit like that. So you break up your app into smaller pieces and the main app calls the various modules and the modules can call even other modules if you want. So everything is nicely organized.
Dashboard structure and layout-as-code
All right. So this is what we wanted. So next here is a screenshot of our dashboard. It's fully in Dutch. Don't try to read it. You might get a headache. It's just here to show the outline of how we use modules to structure the thing. So on the top level, you see on the left-hand side, there are a bunch of pages. And pages are the top level module here. So this is the content of a page. And basically the dashboard consists of a bunch of them. And every page then consists of tiles. These blocks inside a page. And every tile can contain an actual content module, let's say. In this case, there are two. And you see with tabs you can switch between them.
So the whole setup, let's say, of the modular system is that you have a generic module for a page, generic module for a tile, and they are reused for all of them. And then there are more specific modules for the actual content, right? You can reuse them, but, you know, they're a bit more specific. So with this setup, we could actually link that to access control and decide who gets to see what. But we also had to do something about the layout of the dashboard. Because if you imagine you have the same layout for all of your users, but some of the modules are not accessible, either you log in and you see a giant access denied message in certain places on your page, or maybe there are these giant gaping holes where apparently there was a module, but you're not allowed to see it. Also not very nice.
So we had to kind of design layouts for the different user groups that we had. And you can hard code that, of course, in Shiny, but it's way nicer, I think, to define it in a separate way. So we went the JSON route and used a JSON file to define the layouts of the various variants of the dashboard, let's say. So it would look a bit like this. Again, the same page of the dashboard. And then on the left hand, you see the JSON that we use to define what you see. So the top level, again, is the page. And the page has a couple of fields. You see a title and an icon, stuff like that. But there is also a field called modules. And basically, that says which tiles are inside this page.
So if I go over there, expand that. So in the modules field, there is an array of modules of the tile type, let's say. And inside that, again, there is a modules field. And that defines the actual content modules inside. So there's a hierarchical structure there. So if the app is starting up, it doesn't yet know what modules go where. It just has a JSON file and a bunch of Shiny modules that it can load up. And this JSON file, as you saw, contains an array of page objects, really. So we can split that up. And from each of those chunks, a page module is instantiated, which has a modules field, which is an array of modules. And then from that, you can instantiate the tiles. And every tile, again, has a bit of the JSON, the original JSON file, and can start up the content modules.
Communication between modules
All right. So that worked pretty nicely. The only thing that was still missing, well, only a major thing that was still missing, let's say it like that, is communication between these modules. If you ever use Shiny modules, then you know that communicating between modules can be a bit of a pain. But if you don't know what other modules there will even be on a page, how do you communicate with them? I mean, you don't even know if they exist yet. Yeah, that was an issue.
To solve that, here is an example of two modules, let's say a search module and a graph module that need to communicate. And to solve this, we basically double down on the JSON approach. So the search module exports a variable, which is defined in the JSON. So it has an exports field. And the graph module then imports a certain variable. In this case, it imports the org ID variable. And that is mapped to the selected ID variable of the other module using a path. You see it finds the selected ID on page one, tile one in the search module.
So on the R side, it will look like this. If you recall, every module has a UI and a server function. And because it is a function, it has inputs and outputs, which we can use to inject basically the information that we need to get this working. So if we start by the outputs, that's a bit easier. So if in your R code for the search function, you return basically the ID that a user selects, then another module can import that via the inputs and make it work. Why does it work? Well, these two are reactive values, which helps a lot because reactive values are under the hood, like R6 objects, and they are passed by reference, not by value. So if the module starts, you don't get a copy of selected ID at that moment, but you get a reference to the actual variable selected ID inside the other module. So when graph module is running, it actually has sort of a pointer to the original selected ID variable inside the other module. So as soon as someone selects a different, let's say healthcare provider, then reactively things in the graph module will start recomputing because they are reactively linked, like everything should be in a shiny app.
So when graph module is running, it actually has sort of a pointer to the original selected ID variable inside the other module. So as soon as someone selects a different, let's say healthcare provider, then reactively things in the graph module will start recomputing because they are reactively linked, like everything should be in a shiny app.
Runtime customization
All right. Well, with that problem solved, we decided to go one step further. Not that our users asked for it, but it was just too cool not to do, basically, which is to build in a customization option at runtime, because all of these modules are basically loaded at runtime, not at design or coding time. So we could do something like this. Here is a bit of an animation.
So a user can use the side panel and just add the tiles. They can add new modules, which are reactively linked to the ones that are already on the page. Automatically, they can remove modules if they want, resize or drag and drop a bit, just to make it work with their personal workflow. And because we have this JSON file backing all of this up, all of these changes that the user makes are saved to the JSON, so next time someone logs in, you see exactly how you left it last time, and you can just use it however you want to use it. So in this way, we can let people personalize their dashboard. Funny thing, no one asked for it, and almost no one uses it, but it's you know, it's nice.
So this JSON file that we started out with was different per user group, so we had the nice layouts, but in the end, it's actually a personal JSON. So every user who has an account on the dashboard has its own personal JSON definition of their own personal dashboard, basically. All right, so currently, we have a couple hundreds of users on our dashboard from a lot of different organizations in the Netherlands, and instead of the take-it-or-leave-it approach that we have, now we can just say, well, you get that part of the dashboard, you get that module, and people can customize even if they really want to, and make the dashboard fit with their own workflow. And the current dashboard has a lot more content already than the old one ever had, but because of this modular setup, it's actually easier to maintain still, which is very helpful. And because we put in most of the, let's say, the complex part of the coding in the generic modules, like the page and the tile, the actual content modules are pretty clean. So for new team members that join our project, it's actually relatively easy to just start to contribute. It's relatively straightforward to just create a new module, and it will sort of automagically fit within the framework and can be used.
And because we put in most of the, let's say, the complex part of the coding in the generic modules, like the page and the tile, the actual content modules are pretty clean. So for new team members that join our project, it's actually relatively easy to just start to contribute.
So if you have a large, shiny dashboard that needs to show different things for different groups of users, or you want to make it customizable, well, you can do that with Shiny modules. And if you want to know more, I did do a R6-based rewrite of this whole framework. It's a bit different than what I showed you, because it's, well, it doesn't have all the healthcare regulation-specific stuff in there, which we are not allowed to share, obviously. But I think it's interesting, and it has the layout as code part implemented in it, so you can check that out. There's also a link if you want to know more about what we do as a healthcare regulator. And yeah, I think there's room for a question, maybe one or two, because, yeah, that's it. I'm done. Thanks.
Q&A
All right. Yes, I was waiting for the thank you word. All right, so we have a couple of questions. The first one is, they want to know how to add security to a Shiny app, as in embedding the email addresses of the users only, so that if anyone accidentally has access to the link but not part of the users, they will not be able to use it.
Yeah, so for this particular dashboard, we built our own login system, so people log in on the actual dashboard, so that way we know who is who, basically, and then those accounts are linked to user groups, and based on the user group, we know what kind of modules you're allowed to access. Yeah, I think that answers the question, I hope.
How are you storing multiple JSON files for each user in the file folder structure of the application? Oh, that's a good question, yeah. So we use a Postgres database backing up this dashboard, so all of the user accounts and information like that is all in the Postgres database, and Postgres has magnificent support for JSON, so that was the way to go for us. The dashboard itself is on Posit Connect, and then there is a Postgres database behind that.
Last question, how many people do you have working on the application? Can you talk a bit about your team development approach? We have about four developers for the last year or two, I think, working on this. Yeah, we work in an agile way with sprints, et cetera, the whole shebang.