Introduction and background

The use of video is now an established method for delivering information and entertainment on the Web, and its inclusion in Web pages and other electronic materials has been greatly simplified thanks to the introduction of new technology and markup, such as HTML5's video element. Accessible video, while not exactly mainstream, is slowly becoming more available thanks to innovative approaches to delivering captions and video descriptions as well as new regulations and legislation, such as the Twenty-First Century Communications and Video Accessibility Act.

When it comes to making video accessible to blind and visually impaired audiences, the use of video descriptions (also known as audio descriptions) has been the traditional route. New approaches to creating and delivering video descriptions are now in development, such as those which replace human narration with machine-generated, text-to-speech (TTS) narration. For examples of how TTS descriptions can be integrated into Web-based video, see the solutions from IBM-Research Tokyo and the Carl and Ruth Shapiro Family National Center for Accessible Media (NCAM) at WGBH that illustrate effective methods for using HTML5 and Javascript to deliver text-based audio descriptions.

These days, some authors are experimenting with not just passively playing video on a Web page, but instead are launching supplemental materials, or enhancements, on the page in real time while the video continues to play: for example, while a video of a biology lecture about Eastern Cricket Frogs plays on a page, pop-up windows appear at appropriate intervals showing maps of the frog's habitat, or Wikipedia entries about Eastern Cricket Frogs, or pictures of predators. While this can be a useful and exciting way to give students supplemental information in real-time, users who are blind or visually impaired may be completely unaware that these enhancements are opening on the page. These users must not only be alerted to the presence of these on-screen resources, but the resources themselves must be accessible to assistive technology. Take a look at how NCAM uses TTS technology to deliver accessible descriptions, glossaries and Wikipedia entries to make science- or math-focused video materials more accessible to blind and visually impaired users.

With funding provided by the Reader's Digest Partners for Sight Foundation, NCAM has also investigated how to use HTML5, Javascript, MathML and the Popcorn.js HTML5 Media Framework to deliver accessible math alongside video embedded in a Web page. Read on to learn about the demonstration model that is now available for you to test.

System Requirements

To run the demonstration model and access the math via TTS, you will need the following:

Note: While you can play this demo in almost any browser and see the math equations displayed in a region next to the video player, you must use JAWS 12+ and IE9 in order to hear the equations read by a screen reader. (If you don't care about hearing the math and only want to see it, you can use Safari, FireFox or Chrome on both Mac and Windows.) This is an experiment showing how it might be possible for a screen reader to read math that has been dynamically inserted into a live region on a Web page. At this time, however, while some screen readers will read MathML that is displayed statically on a Web page, no screen reader will read MathML in a live region. The demonstration illustrates a temporary workaround solution: the math equations are generated using ASCIIMathML which is converted on the fly to MathML using MathJax. While the sighted user sees the MathML, the screen-reader user actually hears the ASCIIMathML. When screen readers become able to read MathML that has been inserted into a live region, it will become unnecessary to use ASCIIMathML, and authors can simply use MathML as the source for equations, both visual and aural.

Demonstrating Accessible Math with Video

Before playing the demonstration video, please read the system requirements to learn about which screen readers and browsers can be used to take full advantage of the enhancements.

The project's prototype model, Quadratic Equations, demonstrates how accessible representations of math equations can be launched along the timeline of a two-minute video. Each time an equation appears on the screen, the player will automatically pause and a screen reader will read the equation aloud (see the system requirements to learn which browsers and screen readers will work successfully). It will not automatically resume playing, however, so you must resume playback manually using the player controls. The accessible math may be turned on and off using a set of checkboxes, and all of the controls for the player and the enhancements are keyboard and screen-reader accessible. See below for more information on controlling the presentation and the player. The speed, voice and pitch with which the accessible math is delivered will depend on your screen reader's settings.

The player itself can be controlled with a mouse or from the keyboard using the buttons located below the player. Additionally, there are checkboxes located below the player that can be used to select whether or not you want to see and hear the accessible math. Keyboard shortcuts, also known as access keys, have been assigned to each player function. These shortcuts are listed below; your screen reader may also announce them to you.

Keyboard shortcuts for controlling the player

Keyboard shortcuts for turning the accessible math on or off (default is on)

You will need to press certain modifier keys along with the numbers or letters above in order to activate the shortcuts. Which modifiers you press will depend on the browser you're using. A list of browsers and their modifier keys is shown below. (Note: some screen readers will announce the modifier keys for you.)

Keyboard-shortcut modifier keys:

Please see special notes and troubleshooting if you are having problems hearing the enhancements, seeing the video or controlling the player.

Troubleshooting

Screen-reader problems

Can't hear the TTS representation of the math equations with a screen reader; screen reader behaves oddly

  1. First, review the system requirements. You must use JAWS 12+ and IE9 in order to hear the equations read by a screen reader.
  2. If JAWS is not reading the TTS descriptions, reloading the page and shutting down/restarting the screen reader usually causes the descriptions to be read aloud.

Also remember that the speed at which the TTS math equations are delivered depends entirely on your screen-reader settings.

Go to the demonstration now.

Download the source code

If you want to see exactly how we created this demonstration, or want to try implementing it yourself, download a zip file of the entire presentation (21MB).

Your Comments

NCAM is interested in hearing your comments about this demonstration model. Please let us know if you have suggestions for improvements or have problems accessing the TTS enhancements.