Mara Averick & Maya Gans | Data Visualization Accessibility | RStudio Meetup
2:55 - A11Y in R: Adapting Sarah L. Fossheim's 10 dos and don'ts to keep in mind when designing accessible data visualizations | Maya Gans 30:11 - Adventures with {highcharter} and the Highcharts accessibility module | Mara Averick 58:07 - Q&A A11Y in R: Adapting Sarah L. Fossheim's 10 dos and don'ts to keep in mind when designing accessible data visualizations | Presented by Maya Gans Abstract: This talk will use R based visualizations and walk through examples of accessibility considerations when making plots and applications. Speaker Bio: Maya Gans is a Data Visualization Engineer at Atorus Research where she develops custom applications using R and JavaScript. As an RStudio intern she designed TidyBlocks, a visual block based programming language. Maya also co-wrote JavaScript for Data Science. Maya uses `ggplot2` and `d3.js` to create music related infographics for JamBase.com. When Maya's not coding, she's climbing mountains. Adventures with {highcharter} and the Highcharts accessibility module | Presented by Mara Averick Abstract: Lessons learned about accessibility in data visualization through using the {highcharter} R package and the Highcharts visualization library's accessibility module. Speaker bio: Mara is a developer advocate at RStudio. She is the author of neither the highcharter package nor the Highcharts charting library, but enjoys using both to make interactive, accessible data visualizations. So many amazing resources shared yesterday on data visualization accessibility: Sarah L Fossheim's blog - intro to designing accessible data viz: https://lnkd.in/dAAXfE35 Coblis - Color Blindness Simulator: https://lnkd.in/dJT-hJE4 Web Content Accessibility Guidelines (WCAG): https://lnkd.in/dJKmvFTq Color Contrast Accessibility Validator: https://color.a11y.com/ Google lighthouse (automated tool for improving quality of web pages): https://lnkd.in/d8xjSN5i A11y Project Checklist: https://lnkd.in/diFM_TBd Chartability (questions) for ensuring data visualizations, systems, and interfaces are accessible: https://lnkd.in/d_7wk3zx Accessible {highcharter} GitHub repo (Mara's charts, and source .Rmds): https://lnkd.in/dhvBwQ-f Mara's blog post series: https://lnkd.in/d9xz6VZ6 10 Guidelines for DataViz Accessibility by Γystein Moseng: https://lnkd.in/dS-XsKxw Accessible visualization via natural language descriptions by Alan Lundgard and Arvind Satyanarayan: https://lnkd.in/dpdN4skK DataViz Accessibility Advocacy and Advisory Group: https://lnkd.in/d336ACn3 Alt-texts: The Ultimate Guide by Daniel GΓΆransson: https://lnkd.in/dsHcvPs2 JooYoung Seo's Talk on non visual interactions with R packages: https://lnkd.in/dcid56BT Accessible Data Science for the Blind Using R: https://lnkd.in/dTWZbau8 Maya's blog on skip links: https://lnkd.in/dFTYFxTk Twitter alt text: https://lnkd.in/d9bKqiPU Twitter account to follow: @alttextreminder Silvia CanelΓ³n's blog posts: https://lnkd.in/drwbE2Rf Packages shared: ggpattern: https://lnkd.in/dyTBvvz4 gglabeler: https://lnkd.in/dumA8Um8 gghighlight: https://lnkd.in/d_m25j7x sonfiy: https://lnkd.in/dyPwHimP tuneR: https://lnkd.in/dWi2WZH8 brailleR package - https://lnkd.in/d_75cdnQ Caleb brought up a great point that visualizing data isn't new, so it cam be helpful to look at adjacent disciplines to see ways people have solved these before as well. For example, cartographers have had really creative ways to make things visible. Sarah Belle, cartographer who makes fonts / typography really legible on maps: sarahbellmaps.com/belltopo-sans-font-by-sarah-bell/ Cynthia Brewer's work on color palettes. https://colorbrewer2.org/# cartographers who use 3d printing for tactile maps: https://touch-mapper.org/en/
image: thumbnail.jpg
Transcript#
This transcript was generated automatically and may contain errors.
Hi, everyone. Thank you so much for joining us today. Welcome to the RStudio Enterprise Community Meetup. I'm Rachel calling in from Boston. Well, really Dorchester today. Today, we're going to learn more and chat with each other about a crucially important topic, accessibility. So ultimately, making sure that our data products can be used by all users.
If you've just joined now, feel free to introduce yourselves through the chat window. Say hello, maybe where you're calling in from. I do also want to make note that if you want to turn on live transcription during the meetup, you can do so in the zoom bar below where it says more.
To go through a brief agenda, we'll have some short introductions and welcome everyone to the meetup. We'll have a presentation by Maya, A11y in R or ally in R, we were just talking about how you say that. Followed by a presentation by Mara on the high charts accessibility module.
And then a lot of time for Q&A and open discussion to share resources with each other as well.
So after the presentation, we'll have lots of time for Q&A, but can also ask a few right after each of the presentations too. Just a reminder that this meetup will be recorded and shared to the RStudio YouTube. So if you do ask questions in the zoom chat, you'll be part of that recording.
If you want to ask questions anonymously, you can do so as well. We have a Slido link for that, and I'll put that in the chat window here in just a moment.
But for anybody who's joining for the first time, a special welcome to you all. This is a friendly and open meetup environment for teams to share the work that they're doing within their organizations, teach lessons, learn, network with each other, really just to allow us all to learn from each other. So thank you all so much for making this a welcoming community too.
We really want this to be a space where everybody can participate and we can hear from everyone. I always like to reiterate this, that we love to hear from everyone, no matter your level of experience in the field too. I'll share a few other links in the chat window too. If you do ever have suggestions or want to speak at one of these in the future, love to hear from you.
But with that, that's enough from me. Thank you all again for joining us. I'd love to turn it over to our first speaker, Maya Gans. Maya is a data visualization engineer at Atoras Research, where she develops custom applications using R and JavaScript.
Maya's talk: A11y in R
Hey, everyone. I am so excited to talk about this topic today. But a couple caveats first. One, this is a topic that I am super passionate about as of recent. I am not an expert in this, but I think that it is imperative that we all make considerations for accessibility.
The second thing is that I personally, when creating this talk or creating websites, visualizations, whatever it is, I have my own data visualization blind spots and so, yeah. That's just how I'm I just wanted to caveat that. That said, this is a really cool topic and I think we all need to be considering it. As Rachel mentioned, the talk is called A11T or Ally or Accessibility in R. There are 11 letters between A and Y in accessibility and that is long and that is why it is abbreviated as such.
And again, I am not an expert, but Sarah Alfosheim is and their blog posts are incredible, amazing resources. I will be sharing these slides and all the links are clickable therein. And Sarah, their blog post goes over has a blog post where they go over 10 do's and don'ts in data visualization. I figured I could adapt those into the R ecosystem.
Their blog post begins saying accessibility should always be a focus when designing products and the same goes with working with data visualizations and graphs. So, the main tenet of the internet is that everyone should have access to it. And likewise, we should be thinking about data visualizations in the same way. Thinking about our audience, who is consuming them, and they should be consumable by all.
Don't rely on color to explain the data
So, I'll just jump into our first tenet here. Don't rely on color to explain the data. So, I just made a quick bar chart here of where they're grouped by some category. You'll see the code on the left-hand side there. And there are many applications to do this, but here I've linked Cobliss. You can throw in any image into this application and it will show you the various forms of color blindness. And you'll see here, like, this one specifically, you have that middle bar and the top colors are really close together. So, throwing your data visualizations into software like this is a quick and easy win to think about how you're going to leverage different colors.
So, what can we do? There is a really cool package, ggpattern, which this is my first time playing with it. You can, if your company has a really strict color palette that is failing some of these color blind tests, you can leverage different patterns. So, here we have different patterns denoting the different categories as well. So, leveraging this with the redundancy of color can help to bridge that gap.
Don't use very bright or low contrast colors
The second tenet is don't use very bright or low contrast colors. So, I obviously had to make a custom shiny input widget where you can scroll to see the text color on the background color and if it passes the WCAG standards, there are AA and AAA standards, which are the less and more strict. If your website is not passing the less strict one, boom. Bad. So, there's color checkers online, but again, me being extra, I made a little package and made a contrast checker.
Don't hide important data behind interactions
Tenet three. This is a really good one and honestly very fun to play with. Don't hide important data behind interactions. When thinking about accessibility, something that we have to think about is how many consumers of data are doing so on mobile. That is their primary means of getting and consuming data. So, on the left hand side, I have a plotly data visualization where on hover, I'm seeing some drill down. The value of these of the chunks here that make the categories that comprise the whole bar. That's really cool. But on mobile, we don't really have a hover. So, what we can do is leverage click events.
For anyone who wants to geek out over the code, on the left hand side, all I want you to focus on is the hover template. I created a custom hover template in plotly. And then for the click event, which is a little more involved, you have to create that container element. So, I made that container and called it plotly output. So, that's just empty. And then using the HTML widgets package, I attached an on render function for the plotly object where on click, you grab that element and then shove in the data that you get on click, which plotly helps you to leverage that click event.
Don't overwhelm the user with information
Number four. Don't overwhelm the user with information. When you Google dashboard fatigue, I think the first thing that comes up is Sean Lopp of RStudio, his talk on dashboard fatigue and how to mitigate that. What is dashboard fatigue? When you throw it is as a data scientist with a statistical background and not a UI background, it is very tempting to shove all the things into the dashboard because we have access to all the things. But Sean urges us to reel it back and the goal of data science, of a data science team, is to help organizations or individuals make better decisions informed by data. Don't show me everything about everything.
What is it the user needs? Only show them that. And then when designing a dashboard, it's important to think about leading with the key data. Make that bold and font size 100 or make that visual the biggest one. And then two other considerations for not overwhelming the user is to limit the view, the scope, and then let the user, this is, this is like the power of Shiny, right? Where we can let the user change how they filter the data and their focus, slice and dice the data in different ways. Hand that power over to the user rather than just saying, here's 700 plots, have fun.
And then finally, a favorite of mine that I see in the wild a lot is when people don't use consistent language and color. So, example, plot A and plot B are, have the same X axis, but plot A says person 1, 2, 3, and plot B says 1, 2, 3. They're literally the same thing, but you're not using the same, like, nomenclature. Bad.
Use accessibility tools when designing
Number five, use accessibility tools when designing. So, Rachel mentioned that later we can all geek out and share links, because that is the point of these meetups, for us to all learn more. So, the biggest link here is the web content accessibility guidelines. This is the tome of all the things that you should be considering for accessibility when creating a website, and then you can just glean nuggets from that when creating data visualizations.
Google Lighthouse is an incredible tool that you can run on any website. I recommend you, if you're using, like, a package down or blog down or whatever down, run this on your websites, and it will check it gives you a grade if you're and it checks things like the color contrast checker that I created. It checks things like do all your images have all text tags, and it'll walk you through how to make your score 100. So, Google Lighthouse is an incredible tool, and then there's the 11T project checklist, which is a great checklist to mentally or literally go through when creating a data visualization. Make sure that you cross off everything and consider everything outside of your own personal biases when creating a data visualization.
Use patterns and shapes in addition to color
Use patterns and shapes in addition to color. This one is a little we talked about a little bit in point one. Here in the code, I have the color is equal to category, but use some redundancy there. I also added the shape is equal to the category. This might seem redundant to you, but it could be really helpful to others. Simple, effective, just do it.
Using labels and legends
Using labels and legends. So, it is perfectly adequate to use the ggplot2 default legend that is going to create this color by person legend for you, but I urge you to be creative and let your plot readers be as lazy as possible. I love a transition where I get to read this line, and boom, I'm right into person three. I don't need my eyes are doing the least amount of work possible. So, this is just a simple example of how we can use our creativity to leverage labels and legends, and I included the code here because you have to do some shenanigans with shifting the margins in order to add text outside of the plot.
Keyboard navigation and screen readers
All right. This is a slide that is a talk in and of itself, and lends itself to Mara's talk, but there is a library called Chartability, and this is a plot I did not make. It was made by Frank Elvasky. I believe this package is from Visa, but do not quote me on that. This will link out to the observable notebook where this plot was created. In all of my examples till this point, I've been using pngs with alt text to describe the data visualization, and when writing alt text, you want to really think about, if you were to call your friend on the phone and describe the data visualization, that is what you want to put. Don't just put, this is a bar graph. I didn't get anything from that, right?
Don't just put, this is a bar graph. I didn't get anything from that, right? So, what is awesome about the internet is that it uses HTML, which is a hierarchical markdown language.
So, what is awesome about the internet is that it uses HTML, which is a hierarchical markdown language, and also learning HTML is a little bit beyond the scope of this, but I just want to show how this library, because it's not a png, elements are clickable, and here what I did is I clicked, or you can hit tab to focus on the whole plot.
When you hit enter, so HTML is hierarchical, so hitting tab will find sibling elements. Hitting enter, you'll go into the tree, so here I've hit enter to go into the plot itself, and then I can hit enter again, and the screen readers will read to me that this bar segment is of count four, category, current status, and it's about the topics covered, and because this four is a sibling element to the four, I can now hit my left and right arrow keys to navigate to the 41.
This experience of using the keyboard and not using a mouse at all, I urge everyone to do this, so I am very much a mac user, but just going over to a blank slate google page here, if I hit command f5, welcome to voiceover, voiceover speaks descriptions of items on the screen, and can be used to control the computer using only your keyboard.
I am a super voiceover novice. I couldn't even tab from zoom, which is what I'm using into this actual google window, and that is why I urge people to play with this and navigate through using only a keyboard, and at the very least, if you are a shiny app developer, if you can take from this talk to tab through your application and see what that user workflow looks like, that would be a huge win on my end.
So again, I urge people to explore using a screen reader and how that's a new way to experience using a computer for you, as it is for me. It can be incredibly, it's, you want to consider the things that people are going to be tabbing through, maybe leveraging shortcuts in order to make tabbing easier for them, and thinking about keyboard shortcuts and all that kind of stuff is actually really fun. As a side example, if you use a screen reader on twitter, the first thing it reads to you is an element that a sighted user is never going to interact with. It reads out all the keyboard shortcuts to use when navigating through the website, so that you can leverage all the things that someone who's using a mouse could do.
Providing context and explaining your visualization
Number nine is providing context and explaining your visualization, and I, the way that I did this is using the annotate function, which is your best friend here, to add labels. I realize this text is super small, but at the bottom of the plot, I added all time low, and at the highest point in the plot, I added record high. This gives the user some broader context for if this, if you have a graph of some snapshot in time, but this is a record high temperature in the past decade or whatever, like it's, it leveraged text to really highlight those important narratives.
Another example here is I've created a bar plot, and you immediately are going to go to the maroon segment here, because I've leveraged both color and some text, which I very creatively wrote, some event worth highlighting, because that's where your eye is just going to immediately go there. So again, providing context, explaining the visualization, thinking about your audience, what they know, what they don't know, and really trying to communicate that story.
Focus on accessibility during user interviews
And then my last, by my, I mean Sarah's, last tenet here is to focus on accessibility during user interviews. So this applies more to creating dashboards and shiny apps than it does to a single static data visualization, but you still can, all, you should always be applying this first part in the preparation of creating anything.
You should give user personas disabilities. So first of all, what is a user persona? When creating a website or visualization or whatever that you want people that are not yourself to consume, it is really helpful to create detailed narratives of people who are going to be using this application. Sarah, who's the head of HR, blah, blah, blah, blah, she's going to be navigating through the application to do this, and Bob, who's this, is going to be doing that, right? So flushing out those stories will help you to live outside of your subjective biases when creating these applications.
The next step after preparing is recruiting. When getting people to participate in any of your user studies, you want to make sure you set one-on-one time. You might want to consider a ride share or something like that. You don't know what people's transportation needs are. And then finally, you want to inquire about the assistive technology used, which kind of goes into my last one here when you're conducting the interview. If I were to take a poll for all the people in this room, just the different mechanical keyboards that people use, we all have vastly different setups and preferences, and everyone in programming is opinionated, so people are going to have different hardware, so if you create an application with one kind of accessibility hardware in mind, and then someone brings in a joystick, you need to account for that.
And then lastly, you want to just consider that when scheduling these kind of interviews and people work using your application, that these might take longer, because as you saw with the screen reader, there's a lot that goes on there. So again, I'll give out these slides. There's links abound. I hope that I was able to impart some wisdom here, and I hope that if you are new to employing Ally techniques into your data visualization, that you share with us kind of what you learn, and yeah, I'm excited to be here and talk with you all.
Q&A after Maya's talk
Thank you so much, Maya. I always like to say we're all clapping, even though you can't hear us right now. We can use little Zoom reactions, though. Thank you so much for all of that. I see there's a lot of questions that have started to come in through both the chat as well as Slido, so I'm going to do my best to navigate those, but one of the anonymous questions was the principle of contrast checker is also applicable for spatial data map visualizations. Is that true?
Contrast checkers are for like the ratio of contrast for text as opposed to color blindness checkers, which look at what people with different vision situations see. So they're a little bit different, but there are checkers for both, and there are ones that are specifically for maps.
And thinking about outside of your subjective biases, like if there is a gradient that you're using, a default gradient, and you don't know how to create a custom gradient, then leverage all text. Leverage descriptions. You know what I mean? So it's okay if you're not up to the highest standards in WCAG, but then use the other tools if you can. You know what I mean?
That's a great point, Maya. Let me go over to a question from the chat over here on Zoom. I see Nils had asked, are you using a custom shareagen theme?
I live for customizing my shareagen slides, and yes, I am. I use a combination of Canva, creating background images, and importing them into the slides. Talk about extra.
I see Francisco, you just asked a question in the chat if you'd want to unmute yourself and ask live too.
The question was, I wonder if Maya could quickly talk about her process of creating a visualization and adding accessibility layers. Do you follow the principles and order? That's an awesome question. Again, harkening back to my caveat of this being a very new world for me, but I find this incredibly fun and important. In the slide deck, there's a link to a checklist. So, that is what I've recently been considering. And then I use a combination. I always use Lighthouse checks when I'm creating shiny apps to make sure that all my HTML elements are passing everything.
Mara's talk: Adventures with {highcharter} and the Highcharts accessibility module
Hi. Can you hear me? Awesome. Great. So, I am using Quarto, which you'll learn about in future times, or you have already learned about. Anyway, this is my first time doing this with Quarto. And I will say, writing this very brief talk took me way longer than probably any other talk I've ever written. And really not for lack of materials, I did write a five-part series of blog posts covering this topic. But in part because I've fallen down the accessibility rabbit hole, right? There's so much to learn and it's really interesting.
Accessibility is not one size fits all at all. And I ended up, talk about stealing from experts. I'm using Noto Sans, which is what they use on the Ally project and the World Wide Web Consortium on Web Accessibility Initiative for their slides and presentations. If it was good enough for them, it's good enough for me.
Rachel kind of covered this about me. My name is Mara, developer advocate at RStudio. I live in Missoula where it is a balmy 14 degrees right now. Things I did not create. Hi, Charter. That's good old Joshua who is an R package developer. Hi, Charts. That's done by HiSoft.
So, HiCharts, it is a JavaScript charting library based on SVG that makes it easy for developers to create interactive and accessible charts. The SVG part comes in quite a lot. But so, I want to disambiguate what the two things are. And HiCharter is an R wrapper for the HiCharts JavaScript library and its modules.
And how did I fall into this? So, Sylvia wrote a great post on resources for data vis accessibility. And they are these general and R specific resources on how and why to make accessible data visualizations. And at the time of its first writing, the HiCharts accessibility module didn't seem to be working with the HiCharter package. And I used HiCharter before. And I kind of remembered. I was like, you know what? I'm not a JavaScript developer. But if you can make it work in HiCharts, you can usually make it work in HiCharter. So, I decided to see if I could do it.
So, my motivate there were a couple of GitHub issues people had filed. And this was my great motivating example was somebody filed an issue saying that keyboard navigation wasn't working.
And you know what's really fun, by the way? Using keyboard navigation when you're trying to present a slide deck that uses keyboard navigation. But trust me, I am using my keyboard right now. You can see. Look, I did make it work. And look, I can even use my keyboard to do something like view it as a data table. Which turns out people want that. And then I could go back and hide the data. Now I'm using my own mouse. I could export the data. I could export it as SVG. Hooray. I made it work. And this was not the end of the story.
So, if we take my journey of the starting point, reading Sylvia's fantastic post, the end point, reading a five part series on how to do these things with increasingly complex visualizations. The posts have these accessible examples with High Charter and all this R code which is pretty long. But in the middle here is me learning a bunch of useful things. Non-coder framework specific. A lot of great things from High Charter but also things that apply in general. And this talk will be covering that middle point.
Highcharts accessibility features
A little bit of background on High Charts and accessibility. So, one thing which is this is not to say that it's perfect. But High Charts is not free for commercial use. And it's like a key selling point for them. Tons of companies have to comply with accessibility like 508 or regulations in the U.S. And it's a key part of their business model.
So, some of their features very much this is like not at all exhaustive is keyboard navigation, screen reader optimization, low vision features, sonification, voice input, tactile export. So, as you can already probably tell there, that actually accommodates a lot of different types of accessibility needs. So, for example, voice input for people who have trouble using the keyboard. Which is another population. It turns out people who use screen readers have really different preferences.
So, the background on this is that High Charter sorry High Charts did a collaboration with Elsevier. And Ted Gees is from Elsevier. And both of whom they're members of the Data Vis Accessibility Advocacy and Advisory Group. But they kind of got together to Elsevier was actually already using High Charts. And Ted had a lot of background in user testing. And they embarked on this journey of kind of developing this product together through iterative both accessibility expert testing and user testing. And that's really key.
So, this is just one quotation from an article about the project that they did from Lucy Greco who's a web accessibility evangelist. She's one of their test users. And she's been blind since birth. She said, you know, kind of, this innovation allows me to interact with the chart and understand the relationships of all the components of the chart to all the other components rather than just getting a description of the chart.
This innovation allows me to interact with the chart and understand the relationships of all the components of the chart to all the other components rather than just getting a description of the chart.
And that is enabled by the fact that these are SVG. And I full disclosure, like, don't know a ton about SVG. They've made it work. And you can drill down in it. I would never be able to come up with that. I'm not an expert. And I also don't have user testing. So, this is one of those cases where sometimes using the right tool teaches you a lot. And again, accessibility is hard. So, I like using this tool.
So, more background, if you're curious. They did the two of them have presented at the CSUN assistive technology conference is one of the biggest assistive technology conferences they've presented several times. And I really highly recommend this video interview.
There are a lot of links in here that I hope that you take a look at that. But that video interview is one of them. It's short and it really tells you a lot. Other collaborations that HighCharts has done with the Georgia Text Notification Lab, they made a completely open source HighCharts Notification Studio.
Keyboard navigation and screen reader demo
So, keyboard navigation. Users are able to navigate and interact with several components, right? So, the data points, the chart menu, and other chart controls using the keyboard only. So, this kind of goes beyond just being able to get to the chart or around the webpage on the whole. This means that they can interact with individual elements.
And one of the things that they found in their research was that for some users, they preferred using a data table when the data was relatively simple. So, you're able to view data table, which doesn't work out that well on a slide. I have to use my mouse. I apologize. And you can export the data if that's what you want. Again, accessibility is not a one size fits all type of situation.
Sonification. I did not create this one. But here is a cool demo of it. That allows somebody who otherwise wouldn't be able to kind of take in that information for themselves to hear patterns without it being described to them. Which was a feature that people wanted. And I know we had a low vision user accessibility expert who showed his experience in what it was like to get patterns from sonification. And that was incredibly meaningful to him.
I also wanted to show you an example of voice input. I had to embed this so that I can have my closed captions on for you. So, this video demonstrates using Dragon version 15 to navigate interactive features of high charts by announcing the accessible name of buttons and controls. Click JAWS. Click JAWS. Click voiceover. Click voiceover.
All right. So, I'm going to end it there. But that is using voice input to click and drill down on data series. Which is pretty neat.
Screen reader accessibility and the accessibility tree
Screen reader accessibility. Now, this is one part of it. This is for screen reader accessibility for individual points. So, one of the things that they found, again, with user studies was that rather than having these really long descriptions, a lot of users preferred that they get the information only when they need it.
And I apologize if that was more video situation. So, this allows a user who can use their keyboard to hear what is happening at each point. One. December 2010, 34.8%. NVDA. One. December 2010, 20.2%. Voiceover. Two. May 2012, 30.7%. Voiceover. Three. January 2014.
Cool. And oh, by the way, you might note here, and this will come up more later, is that this is actually these are the most common desktop screen readers. And they as you'll find out as I continue, they do not all they all have platform specific and sometimes browser specific accessibility APIs. So, when you're testing something, it's not as easy as being like, use a screen reader. They actually all work a little bit differently. So, testing is pretty extensive.
Why is it so impressive? Well, this gets at what I was talking about before. So, assistive technology and the accessibility tree. So, this is an example with an application. So, an application is broken down into something called an accessibility tree. And those nodes can be grouping nodes that are hierarchical or they can be individual elements that you can interact with in different ways. So, it could be something that you could click, that could have a state, that could have a role. And again, this is a platform specific accessibility API that creates that accessibility tree. And then that goes from the assistive technology to the user.
So, an example, this would be with using VoiceOver. The app translates an element into an accessibility node. VoiceOver visits it. VoiceOver announces what it is. Then the user, in this case, would be using the keyboard. They can press it. VoiceOver tells them what they did. The click event is routed back to the UI element and something happens.
Turns out it's pretty similar on the web. So, in the browser, and browsers vary from in terms of what they do, this pretty much goes to the DOM, the document object model, the tree that is created for us. And what screen readers see is controlled by their accessible written internet application roles. That's ARIA and the HTML. So, it's in parallel. The tree is painted into a visual UI and transformed into an accessibility tree, which varies by browser, which is then read via the native API for the assistive technology and brought to the user. So, this is, like, not a simple thing to solve.
What Mara learned from using the accessibility module
So, now I'll talk a little bit about what I learned from using the accessibility module. This is a chart that I created in our using high charter. I'm not showing you the code because it's super long, but don't worry. It's in the source documents. It's navigable by keyboard. I can export. I have an annotation.
So, what's the magic here? Here's where some of the screen reader cool stuff comes in. And Maya, I think, mentioned this before with Twitter. So, if you go and you inspect the HTML for this, it's actually created a hidden screen reader region that if you look at, it's ARIA hidden equals false. So, that means that it's visible to a screen reader. And then there are other regions of the chart that are hidden, but it is visually hidden to a user who's looking at a visual representation. And then this back from that first article, this is where Ted and the developers added the structure description of a chart that could benefit screen reader users, such as the chart type, access information, which is partially automatically generated and a long description of what's found in the chart provided by the creator.
So, again, it's not one size fits all, right? Part of what they found was that users, they wanted not to have to go through every element in order to get the description. Or they wanted to be able to skip certain elements early on. And this is a case where you have two different ways of accessing similar information that accommodates different users.
So, the default format for that screen reader section is... So, the chart title, basically, the type description, what kind of chart it is, chart subtitle, chart long description. I will get back to that in a moment. Apply a sound button if you have soundification disabled. The view table button.
That view table button comes up pretty quickly. And that's because for simple data structures, a lot of people preferred that. And the chart long description, some people don't like that. And there's an easy way for them to skip it. So, again, making different affordances for different personas.
So, for that penguins chart I showed you before, what does my screen reader section look like? Again, title and chart info. Flipper length versus bill length. That's the title that I entered. And it tells you what kind of chart it is, a scatter chart, how many data series, three. And then a subtitle that I added myself. Axis descriptions and ranges.
This is a combination of auto generation and user generation. So, for example, when I wrote this code, I the X axis description, I specifically wrote out displaying flipper length in millimeters knowing that it would be in this special area. And that they would be able to then skip the X axis description elsewhere. The range can either be automatically generated, which is what I did in this case, or the developer, the creator can write it in there.
And then the annotations. And this is a big deal that I didn't really know about. And I'm showing you an example that just has one annotation, but there's another one in the blog post that has lots of them, is that the chart annotation summaries come up. Because often with annotations, we're saying what the most important information is.
The chart long description and semantic content
So, I skipped over the chart long description before, where you are most irreplaceable. You're not irreplaceable at any point. But this is where you can say more. And what belongs there? Well, it depends. And it depends on what? Well, a couple of things. One is both the intention of the chart. So, what you're trying to convey. And it depends on whom you ask. So, in the user studies, as I mentioned before, they found that it was important that it be skippable. But in some other research, which wasn't done by high charts that came out, oh, what belongs there in general? A text summary of the visualization, making sure to describe trends or patterns in the visualization.
But different kinds of description contain different kinds of content. And this was a study done recently about a four level model of semantic content in different descriptions. These four levels, the first one is kind of the stuff that I covered automatically. So, the elemental and the encoded properties, that's things like title, legend, chart type, what are the ranges? Second level was statistical concepts and relations. So, things like computable descriptive statistics. And now, as there's statisticians here, you might start to think, like, wow, I definitely don't want a computer automatically saying the descriptive statistics that are the wrong statistics for the kind of for the distribution I'm showing. You know, you can see how human intervention would be useful there. The third level was perceptual and cognitive phenomena, which basically means stuff conveying kind of the overall gist of complex trends and patterns, exceptions, outliers. And then the fourth level was contextual and domain specific insights. So, like, social and political explanations for trends and stuff that depends on an individual's reader's subjective knowledge.
And some of the interesting takeaways from this study were that description, so, the study was done with both blind and sighted reader groups. And they found that descriptions are important to both blind and sighted reader groups. But that those reader groups differ, almost diverged, significantly on which semantic content they ranked as most useful. So, not everybody says that the same stuff is useful. And this was interesting to me, because there you're really saying, like, okay, what you think as a sighted person might be most useful is might be very different from what a blind user thinks. And then even within that, access to meaningful information is strongly reader specific.
Lessons learned
All right. So, that was kind of, and I really, really recommend you check out this paper. It's very, very good. Running out of time. So, lessons learned through using this API. Use tools. They're great. I learned a ton from using this tool. And stuff that I wouldn't have thought of. And different tools make different things easy or harder. And you learn a lot. But don't rely on tools. So, there are limitations to technical fixes. Accessibility isn't one size fits all. In fact, what helps one user could be a hindrance to another. So, things like patterns for dual encoding can be really useful to some people and can be really distracting or even harmful to others. And you need to make affordances for those user preferences.
Look to experts. There are people who are experts in this. But it is kind of a nascent field. There aren't really great WCAG guidelines on semantics for things like SVG. Get feedback. Get feedback from lots of different Maya covered this nicely. And keep learning. I think that's one of the things I've learned. Literally since I did this, the high charts API has changed and improved. And this it really is kind of a young area of study.
So, these you can access in my slides. These are has the R code and the working examples. The blog post series, which if you're interested in, you can see them. Resources. The 10 guidelines for data accessibility. One of the high charts core developers. That paper I mentioned earlier. The accessible visualization via natural language descriptions. The data accessibility and advocacy group has an ever growing collection of resources. This really great piece I read by a low vision user who uses screen reader every day on alt text. It's really short and one of the best things I've read about it. It was all these things I as a sighted user would never think of.
And then the ally project is a great meta resource for more resources. And that is it.
Q&A
Thank you so much, Mara. Thank you for all those resources at the end, too.
I see a few questions on Slido that have come through and in the Zoom chat. But I do want to just bring everybody in to the conversation, too. So, I know this is kind of hard on Zoom. But if you want to jump in and ask questions, too, you can raise your hand on Zoom.
Mara, I see David had asked a question on Zoom. Does anyone know a way to add sonification that isn't with high charts? And I think there were a few recommendations of packages, too.
I know there are ways. And they don't all use high charts. I think one of the Let me get back to you on that. I haven't used the other ways. But there are other ways. And that sonification studio isn't actually part of high charts. You can use it for anything. I know there's an alternative way, too. Yeah, I think Riva shared Sonify and TuneR packages that you can use.
Let's see. I see, Zach, you have your hand raised. Do you want to jump in live?
Yeah, I can do. So, this one's for Maya. So, about the clicking on the Shiny app, is that would I have I've kept a ggplot2 at the moment. Would I have to learn Plotly or is there a ggplot2 way of doing it?
Have to is a really funny thing. Yes, you have to. No, you can totally use ggplot2 outputs. Again, it gets into, like, creating alt text for those outputs. And I don't know if anyone has played around with or done research on, like, when you dynamically create a plot in Shiny, also have alt text come along for the ride with it. I personally, like, my journey geeking out into accessibility is very young, very green in the space. But I'm super excited about trying to automate some level of description. Mara talked about the dangers in doing that. But also, like, the baseline of having nothing. Like, it's better than that.
Theoretically, if you render it as an SVG, then you could create events on those SVGs if you made it so that you could focus on each one of those. Then you would have ARIA roles that you would have to manually assign to them, saying how they were related back to the chart. So, where they come from. And then the data that is there's no standard for this. But the data that is with it. That's where I'm, like, huh. Boy, would it be easier if I had a tool that already did this for me. Like, that's where I'm, like, I don't know how to do that. But it's possible.
Yeah. Because that's a big thing that I took away was, yeah, a lot of my, a lot of stuff I commented on earlier. I use a red, amber, green. Because, of course, everyone uses a rag system. Red and green aren't the most accessible for color blindness. That was a thing that I picked up on. So, yeah. Thank you very much, both of you. But you can use text with that, too. Like, for example, in the Shiny widget I had, if you picked, like, blue text on a blue background, the box turned red. But under it, I had that with an X on it, you know? So, because that's, like, traffic-lighty norm, you want to leverage things that people see in the real world, but also you want, you can, like, how Mara talked about dual encoding, thinking about just giving users, like, opting into different ways of interacting with the data.
Thanks, Zach. I see Al on Zoom asks, for a user who is blind or partially sighted and has never seen a scatter plot, is it helpful to tell them that it's a scatter plot? What will they do with that information? Also, how do these charts handle screen zoom or magnifying?
Yeah, so, high charts deals with zoom and magnifying. That's, like, another one of their big features. And mobile. Again, I'm, like, great, I don't have to build anything like that. And, yeah, so, that is actually in that paper on semantic content, one of the recommendations, they're, like, here's where you could go into detail about what it means to be a scatter plot, you know? And so, that's one option. And that's also where you kind of want to gauge your audience. Is saying the chart type relevant? Well, if you've never seen it before, if you have seen it before, I don't think that it might say something about how you'll end up navigating it if you become familiar with it. So, for example, for a scatter plot, you're going to want to be going between groups and zooming in. But, yeah, it's, like I've said many times before, what's useful for one user isn't necessarily useful for another one, so.
Just adding to that, too, if you, like, let's say we have radio buttons where the user can toggle between seeing a scatter plot or a line plot, then you need some indicator for an ARIA role that that, okay, I'm seeing the thing that I clicked in the other thing, you know? So, it just ties, it could tie it back in that way, too.
Great. Thank you. I see Caleb had asked on Zoom earlier, is the annotate function built into ggplot2? I think that was during your talk, Maya. Yeah, there's a million packages that people that are smarter than me have written, as well. Like, there's, it's escaping me. I'm sure someone could throw it in the chat. Like, a labeler that will label all your points, and you can even set, like, if-else statements to only label things you care about.
ggplot, right, yeah. Thank you. And there's, and gglabeler. There you go.