Describing Images for Enhanced Assessments
Project Backgroud and Components
Overview
Project Partners
Project Components
Outcomes
Dissemination
Overview
The Description Enhanced Assessments (DEA) project examined the use of description as an assessment accommodation for students with visual and print disabilities, by investigating student comprehension under multiple conditions and documenting meaningful and effective practices for access to visual and complex images within high stakes assessments.
[back]
Project Partners
This project was a multi-state collaboration with participation from the Utah State Office of Education (USOE), Colorado Department of Education (CDE), and Kansas State Department of Education (KSDE). The project also included technical expertise provided by the WGBH National Center for Accessible Media (WGBH NCAM), the only organization to develop research-based practices regarding the use of description with persons with disabilities; the National Center on Severe and Sensory Disabilities (NCSSD) at the University of Northern Colorado, which provides training and technical assistance to educators and has been involved in description research; and an advisory panel of national experts in assessment of students with visual and print disabilities.
[back]
Project Components
The project had four essential components:
1. The first essential component of this project was training for the project partners. This training was related to research, test design, described media, tactile graphics, and visual bias. As project staff members were trained across these diverse areas of expertise, the field gained increased capacity to provide appropriate accommodations for the instruction and assessment of students with disabilities. Project partners participated in training on the use of audio description, potential implications for the use of audio description as an accommodation for assessment, quality and use of tactile graphics, bias and sensitivity review of test items, review of test items for bias as relates specifically to students who are blind or visually impaired, assessment design, principles of test item writing, and low-incidence research methodologies. These training activities were implemented through on-site meetings, conference call meetings, and webinars.
2. The second essential component of this project was the development of an assessment tool for administration to student participants. The assessment used items selected from the Utah Test Item Pool Server (UTIPS). UTIPS is an online formative assessment tool with non-secure test items. A bank of 340 items was culled from the UTIPS collection and subjected to a rating and review rubric that included face validity, alignment of standards across all three states and the Common Core State Standards, appropriateness for production in tactile graphic format, and appropriateness for production in audio description. The final assessment included three items at each grade level 3-8 under the content areas of reading/language arts, math, and science for a total of 54 items. Descriptions for these items are developed based on the NCAM quality indicators.
3. The third essential component of this project wad the administration of the assessment to measure comprehension and evaluate efficiency, clarity, and preferences of students as relates to the use of audio description. The assessment package included an interview for the student and the teacher about description preferences. The assessment was administered to 302 students in grades 3 – 8 in the content areas of reading/language arts, math, and science. Of the 302 students, 120 had visual disabilities and 182 had print disabilities. Analysis of assessment results revealed several significant findings:
4. The fourth essential component of this project was the production of guidelines for best practices in description of test items for national dissemination. Formal research on how to best describe images for assessments is just beginning. However, by applying what is already known about image description and test accommodations, there are ways to improve the accessibility of assessments that can be applied immediately. Guidelines for using image description as an accommodation for assessment can be organized in four categories. These guidelines are published on this website and are the central focus of project dissemination.
[back]
Outcomes
This project resulted in significant outcomes that endure beyond the scope of the project period: student assessment data that contributes to the research base on accessibility of test items to meet the diverse needs of students with visual and print disabilities; capacity-building within partner states to provide consistent, efficient, meaningful, and cost-effective methods of providing access to complex images in test items; and guidelines for widespread dissemination to assist other states in developing description accommodations for their statewide assessments.
[back]
Dissemination
Dissemination efforts focus on promoting awareness and use of quality description as an instructional accommodation. This is achieved through development of a webinar to support teachers in understanding the effective use of description as an accommodation, on-site training on the use of description as an accommodation for teachers in participating states, and communication with assessment developers and the Race to the Top Assessment Consortia regarding the use of description as an accommodation for assessment. Presentations at national and international conferences, written publications, and professional networking also play important roles in sharing project results to increase the use of description as an accommodation for students with visual and print disabilities.
[back]
Funding for this project is from the U.S. Department of Education to the Utah State Office of Education under Grant Award #S368A090019. Any opinions, findings, and conclusions or recommendations are those of the project team and do not necessarily reflect those of the U.S. Department of Education.
Project Partners
Project Components
Outcomes
Dissemination
Overview
The Description Enhanced Assessments (DEA) project examined the use of description as an assessment accommodation for students with visual and print disabilities, by investigating student comprehension under multiple conditions and documenting meaningful and effective practices for access to visual and complex images within high stakes assessments.
[back]
Project Partners
This project was a multi-state collaboration with participation from the Utah State Office of Education (USOE), Colorado Department of Education (CDE), and Kansas State Department of Education (KSDE). The project also included technical expertise provided by the WGBH National Center for Accessible Media (WGBH NCAM), the only organization to develop research-based practices regarding the use of description with persons with disabilities; the National Center on Severe and Sensory Disabilities (NCSSD) at the University of Northern Colorado, which provides training and technical assistance to educators and has been involved in description research; and an advisory panel of national experts in assessment of students with visual and print disabilities.
[back]
Project Components
The project had four essential components:
- train partners in research-based descriptive practices
- analyze, develop and field-test descriptions using formative test items from the Utah Test Item Pool Server (UTIPS)
- conduct student assessments to evaluate description efficiency, clarity, and preferences
- produce guidelines for best practices in description of test items for national dissemination
1. The first essential component of this project was training for the project partners. This training was related to research, test design, described media, tactile graphics, and visual bias. As project staff members were trained across these diverse areas of expertise, the field gained increased capacity to provide appropriate accommodations for the instruction and assessment of students with disabilities. Project partners participated in training on the use of audio description, potential implications for the use of audio description as an accommodation for assessment, quality and use of tactile graphics, bias and sensitivity review of test items, review of test items for bias as relates specifically to students who are blind or visually impaired, assessment design, principles of test item writing, and low-incidence research methodologies. These training activities were implemented through on-site meetings, conference call meetings, and webinars.
2. The second essential component of this project was the development of an assessment tool for administration to student participants. The assessment used items selected from the Utah Test Item Pool Server (UTIPS). UTIPS is an online formative assessment tool with non-secure test items. A bank of 340 items was culled from the UTIPS collection and subjected to a rating and review rubric that included face validity, alignment of standards across all three states and the Common Core State Standards, appropriateness for production in tactile graphic format, and appropriateness for production in audio description. The final assessment included three items at each grade level 3-8 under the content areas of reading/language arts, math, and science for a total of 54 items. Descriptions for these items are developed based on the NCAM quality indicators.
3. The third essential component of this project wad the administration of the assessment to measure comprehension and evaluate efficiency, clarity, and preferences of students as relates to the use of audio description. The assessment package included an interview for the student and the teacher about description preferences. The assessment was administered to 302 students in grades 3 – 8 in the content areas of reading/language arts, math, and science. Of the 302 students, 120 had visual disabilities and 182 had print disabilities. Analysis of assessment results revealed several significant findings:
- Braille readers were more likely to select the correct answers when given image description without tactile graphics.
- All other students in the study were equally likely to select the correct answer whether given image description or not.
- Image description is an unbiased accommodation.
- Teachers reported strong belief in the potential effectiveness of image description.
- Students reported strong preference for image description.
4. The fourth essential component of this project was the production of guidelines for best practices in description of test items for national dissemination. Formal research on how to best describe images for assessments is just beginning. However, by applying what is already known about image description and test accommodations, there are ways to improve the accessibility of assessments that can be applied immediately. Guidelines for using image description as an accommodation for assessment can be organized in four categories. These guidelines are published on this website and are the central focus of project dissemination.
[back]
Outcomes
This project resulted in significant outcomes that endure beyond the scope of the project period: student assessment data that contributes to the research base on accessibility of test items to meet the diverse needs of students with visual and print disabilities; capacity-building within partner states to provide consistent, efficient, meaningful, and cost-effective methods of providing access to complex images in test items; and guidelines for widespread dissemination to assist other states in developing description accommodations for their statewide assessments.
[back]
Dissemination
Dissemination efforts focus on promoting awareness and use of quality description as an instructional accommodation. This is achieved through development of a webinar to support teachers in understanding the effective use of description as an accommodation, on-site training on the use of description as an accommodation for teachers in participating states, and communication with assessment developers and the Race to the Top Assessment Consortia regarding the use of description as an accommodation for assessment. Presentations at national and international conferences, written publications, and professional networking also play important roles in sharing project results to increase the use of description as an accommodation for students with visual and print disabilities.
[back]
Funding for this project is from the U.S. Department of Education to the Utah State Office of Education under Grant Award #S368A090019. Any opinions, findings, and conclusions or recommendations are those of the project team and do not necessarily reflect those of the U.S. Department of Education.