Effective Practices for Description of Science Content within Digital Talking Books
Project History and Research Methodology
PREVIOUS | NEXT | CONTENTS
WGBH's National Center for Accessible Media (NCAM) received a National Science Foundation (NSF) grant to research and document effective practices for providing meaningful descriptions of non-text science content for post-secondary students or scientists who have vision loss, in collaboration with the American Foundation for the Blind (AFB). The project united staff from NCAM and AFB with researchers and experts from Recording for the Blind and Dyslexic, Inc. (RFB&D); the American Printing House for the Blind (APH); scientist and former NSF program officer Lawrence Scadden and DAISY Consortium Secretary General George Kerscher in a collaborative effort to develop research-based guidelines that will inform provision of effective descriptions for science content in future digital talking books (DTBs). It is significant that this project has the support and participation of the DAISY Consortium, the world-wide network of libraries, publishers and service organizations that developed the DTB standard. All project partners contributed to the DAISY/NISO standard and all partners were among the 40 members of the National File Format Panel, which produced the National Instructional Materials Accessibility Standard (NIMAS).
Over the past four years, the project has conducted two rounds of a web-based Delphi survey with 30 participants, who were either describers or students/scientists with vision loss, in order to seek a consensus among experts on approaches to the description of STEM charts, graphics, tables, and other non-textual content. AFB conducted the analysis of data from these two surveys.
The final phase of the project was a 60 person end-user survey which evaluated the practices suggested by the two rounds of the Delphi survey. The intent of the end-user evaluation was to corroborate agreed upon approaches, evaluate preferences between near-ranked but alternate approaches, and explore areas of disagreement on effective approaches that might emerged from the Delphi surveys.
End-users were recruited through announcements on the AFB web page, distribution to email listservs maintained by the American Council of the Blind, the National Federation of the Blind, and AFB's CareerConnect, as well as informal referrals from AFB and WGBH staff.
Participants provided demographic information that the Delphi surveys suggested were important to the analysis of data, including degree level, current age, field of study, how long in the field, current and preferred access strategies, and frequency of use of Daisy Talking Books. The second Delphi survey found a difference among respondents depending on when print alternatives were first utilized, so this information, as well as which print alternatives the potential participant used, was also included.
Sixteen (16) images were included in the survey. Images were balanced across subjects through random presentation, to eliminate any possibility of an order effect. Each image requested participants to rank at least the clarity and efficiency of the description on a four-point scale:
Participants were also asked to add specific comments about ways to improve the descriptions "and other techniques you have used to access similar images."
View a sample from the end-user survey.
We hypothesized that the relative novelty of electronic descriptions of STEM images might affect participants' rankings of the clarity and efficiency of the descriptions, and therefore decided to randomly select participants to receive a tactile supplement consisting of three images from the survey. Two sets of 3 different images each were prepared by the American Printing House for the Blind and mailed to AFB for distribution to 30 randomly selected participants. One half (n = 15, 50.0%) of the participants who received tactile supplements eventually completed the survey.
In general, there was a great deal of agreement regarding the clarity and efficiency of the descriptions corroborating the results of the Delphi surveys. Participants who received tactile supplements did not respond significantly differently than did those who listened to the online descriptions only. The specific results of this research make up the bulk of this website, specifically the Guidelines and the subsequent examples.
Fifty-four (54) individuals participated in the end-user study, although two of these sometimes skipped items. They were nevertheless included in the data analysis. Thirty-one (31) different fields of study were identified by participants and recategorized into the sciences, technology, engineering, mathematics, and all other fields. Analysis revealed no significant differences based on Field of Study.
Unlike the Delphi surveys with experts, the end-user participants were a younger group, with 66.1% between the ages of 25-54. More than half (60.4%) of the participants held Bachelor's degrees. Almost half (49.0%) had 10 years or more of experience in their fields of study. The majority (52.8%) of participants had used Digital Talking Books more than 5 times. The largest group of participants (45.1%) began using alternatives to ordinary print in elementary school or earlier, although more scientists (60.0%) and engineers (66.7%) did not do so until their college years and later.
Participants demonstrated no clear preference for method of reading. When asked their preferred method of reading STEM materials, the same variation was evidence, although a larger proportion seemed to prefer braille (26.4%). Auditory methods (audio book, text via text-to-speech, and Digital Talking Books) were utilized by 73.1% of participants for reading, and by 51.1% of participants for accessing STEM materials. Participants' preferred method for accessing mathematical equations also varied, from Nemeth Code to LaTex to various forms of text-to-speech to human readers.
As each Delphi and end-user survey was completed, the participant was sent a $75 gratuity, along with a personal note of appreciation.
PREVIOUS | NEXT | CONTENTS
Funding for this project is provided by the National Science Foundation.
Project History and Research Methodology
Adults with vision loss who pursue careers in science are confronted with barriers to visual information in science, technology, engineering, and mathematics (STEM) textbooks and journals, which are rife with complicated images, charts, and data tables. Descriptive information about critical content - images, diagrams, and charts - is often omitted or provided inconsistently by volunteer narrators or Braille transcribers who may have limited expertise in STEM content. In most cases, post-secondary students and scientists rely on the services of assistants to read and describe images in order to stay current with content in their fields of study. This does not provide independent access to content, and it is an inefficient, expensive, and time-consuming solution. Further, the majority of adults who are blind lose their vision as adults. Typically, they rely on Talking Books; some may not even learn Braille.WGBH's National Center for Accessible Media (NCAM) received a National Science Foundation (NSF) grant to research and document effective practices for providing meaningful descriptions of non-text science content for post-secondary students or scientists who have vision loss, in collaboration with the American Foundation for the Blind (AFB). The project united staff from NCAM and AFB with researchers and experts from Recording for the Blind and Dyslexic, Inc. (RFB&D); the American Printing House for the Blind (APH); scientist and former NSF program officer Lawrence Scadden and DAISY Consortium Secretary General George Kerscher in a collaborative effort to develop research-based guidelines that will inform provision of effective descriptions for science content in future digital talking books (DTBs). It is significant that this project has the support and participation of the DAISY Consortium, the world-wide network of libraries, publishers and service organizations that developed the DTB standard. All project partners contributed to the DAISY/NISO standard and all partners were among the 40 members of the National File Format Panel, which produced the National Instructional Materials Accessibility Standard (NIMAS).
Research into Current Practices and Delphi Surveys
Over the past four years, the project has conducted two rounds of a web-based Delphi survey with 30 participants, who were either describers or students/scientists with vision loss, in order to seek a consensus among experts on approaches to the description of STEM charts, graphics, tables, and other non-textual content. AFB conducted the analysis of data from these two surveys.
End-User Survey
The final phase of the project was a 60 person end-user survey which evaluated the practices suggested by the two rounds of the Delphi survey. The intent of the end-user evaluation was to corroborate agreed upon approaches, evaluate preferences between near-ranked but alternate approaches, and explore areas of disagreement on effective approaches that might emerged from the Delphi surveys.
End-users were recruited through announcements on the AFB web page, distribution to email listservs maintained by the American Council of the Blind, the National Federation of the Blind, and AFB's CareerConnect, as well as informal referrals from AFB and WGBH staff.
Participants provided demographic information that the Delphi surveys suggested were important to the analysis of data, including degree level, current age, field of study, how long in the field, current and preferred access strategies, and frequency of use of Daisy Talking Books. The second Delphi survey found a difference among respondents depending on when print alternatives were first utilized, so this information, as well as which print alternatives the potential participant used, was also included.
Sixteen (16) images were included in the survey. Images were balanced across subjects through random presentation, to eliminate any possibility of an order effect. Each image requested participants to rank at least the clarity and efficiency of the description on a four-point scale:
- Extremely unclear or inefficient
- Somewhat unclear or inefficient
- Somewhat clear or efficient
- Extremely clear or efficient
Participants were also asked to add specific comments about ways to improve the descriptions "and other techniques you have used to access similar images."
View a sample from the end-user survey.
Tactile Supplements
We hypothesized that the relative novelty of electronic descriptions of STEM images might affect participants' rankings of the clarity and efficiency of the descriptions, and therefore decided to randomly select participants to receive a tactile supplement consisting of three images from the survey. Two sets of 3 different images each were prepared by the American Printing House for the Blind and mailed to AFB for distribution to 30 randomly selected participants. One half (n = 15, 50.0%) of the participants who received tactile supplements eventually completed the survey.
Results
In general, there was a great deal of agreement regarding the clarity and efficiency of the descriptions corroborating the results of the Delphi surveys. Participants who received tactile supplements did not respond significantly differently than did those who listened to the online descriptions only. The specific results of this research make up the bulk of this website, specifically the Guidelines and the subsequent examples.
Demographic Information About End-User Participants
Fifty-four (54) individuals participated in the end-user study, although two of these sometimes skipped items. They were nevertheless included in the data analysis. Thirty-one (31) different fields of study were identified by participants and recategorized into the sciences, technology, engineering, mathematics, and all other fields. Analysis revealed no significant differences based on Field of Study.
Unlike the Delphi surveys with experts, the end-user participants were a younger group, with 66.1% between the ages of 25-54. More than half (60.4%) of the participants held Bachelor's degrees. Almost half (49.0%) had 10 years or more of experience in their fields of study. The majority (52.8%) of participants had used Digital Talking Books more than 5 times. The largest group of participants (45.1%) began using alternatives to ordinary print in elementary school or earlier, although more scientists (60.0%) and engineers (66.7%) did not do so until their college years and later.
Preferences for Reading
Participants demonstrated no clear preference for method of reading. When asked their preferred method of reading STEM materials, the same variation was evidence, although a larger proportion seemed to prefer braille (26.4%). Auditory methods (audio book, text via text-to-speech, and Digital Talking Books) were utilized by 73.1% of participants for reading, and by 51.1% of participants for accessing STEM materials. Participants' preferred method for accessing mathematical equations also varied, from Nemeth Code to LaTex to various forms of text-to-speech to human readers.
Remuneration
As each Delphi and end-user survey was completed, the participant was sent a $75 gratuity, along with a personal note of appreciation.
PREVIOUS | NEXT | CONTENTS
Funding for this project is provided by the National Science Foundation.