Rethinking Accessibility Using NAEP Process Data: Exploring Universal Design and Accommodations

This project seeks to systematically explore the 2017 NAEP Grade 8 mathematics process data to provide empirical evidence on the utilization of AFs, including accommodations and UD elements; how item characteristics, student characteristics, and school characteristics are related to the utilization of AFs; and how AF utilization relates to students’ test-taking behavior and performance. The results from this study will help improve the validity of results from such DBAs for SWD and inform ongoing test development, with a focus on minimizing the impact of construct-irrelevant factors on student achievement in a DBA. The results of this study can be translated into actionable policies (e.g., decisions on use of UD elements), guidelines (e.g., conditions under which accessibility features may benefit students), and tools for school administrators and teachers. In this exploratory study, project team will create novel and multidimensional measures of AF use and employ traditional analyses and emerging techniques in machine learning to explore AF utilization patterns. They will also employ quasi-experimental methods to examine the relationship between AF use and students’ performance and test-taking behaviors. The findings will be disseminated to diverse key audiences, including teachers; district and school administrators; district, state, and federal policymakers; test developers; and researchers. The products of this project will include academic journal articles, practitioner-oriented briefs, conference presentations (general research conferences, conferences focused on SWD, conferences on advanced analytics), social media posts, webinars or virtual online sessions, and publication venues allowing for interactive results using R. As the proposed co-Investigator for the project, Dr. Yin will provide guidance on the analyses and will lead dissemination activities, including the production of peer-reviewed articles, conference presentations, and briefs. Dr. Yin will devote 5.25% of her time in Year 1 and 9% in Years 2 and 3 to the proposed study. In Year 1, Dr. Yin will participate in bi-weekly project meetings, provide scientific guidance on analyses, and provide review of manuscripts, reports, and other related products. In Year 2, Dr. Yin will participate in bi-weekly project meetings, participate in manuscript and report development, and lead dissemination activities. In Year 3, Dr. Yin will participate in bi-weekly project meetings, lead dissemination activities, including journal article development and submission, conference presentations, and other related activities.