Abstract
Objective: To develop a reliable and valid instrument to assess the understandability and actionability of print and audiovisual materials. Methods: We compiled items from existing instruments/guides that the expert panel assessed for face/content validity. We completed four rounds of reliability testing, and produced evidence of construct validity with consumers and readability assessments. Results: The experts deemed the PEMAT items face/content valid. Four rounds of reliability testing and refinement were conducted using raters untrained on the PEMAT. Agreement improved across rounds. The final PEMAT showed moderate agreement per Kappa (Average K= 0.57) and strong agreement per Gwet's AC1 (Average. = 0.74). Internal consistency was strong (α= 0.71; Average Item-Total Correlation. = 0.62). For construct validation with consumers (n= 47), we found significant differences between actionable and poorly-actionable materials in comprehension scores (76% vs. 63%, p< 0.05) and ratings (8.9 vs. 7.7, p< 0.05). For understandability, there was a significant difference for only one of two topics on consumer numeric scores. For actionability, there were significant positive correlations between PEMAT scores and consumer-testing results, but no relationship for understandability. There were, however, strong, negative correlations between grade-level and both consumer-testing results and PEMAT scores. Conclusions: The PEMAT demonstrated strong internal consistency, reliability, and evidence of construct validity. Practice implications: The PEMAT can help professionals judge the quality of materials (available at: http://www.ahrq.gov/pemat).
Original language | English (US) |
---|---|
Pages (from-to) | 395-403 |
Number of pages | 9 |
Journal | Patient education and counseling |
Volume | 96 |
Issue number | 3 |
DOIs | |
State | Published - Sep 2014 |
Funding
The information upon which this publication is based was performed under Contract #HHSA290200900012I, TO 4 “Improving EHRs Patient Education Materials” funded by the Agency for Healthcare Research and Quality (AHRQ), Department of Health and Human Services . The content of this publication does not necessarily reflect the views or policies of the Department of Health and Human Services, nor does the mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. The author assumes full responsibility for the accuracy and completeness of the ideas presented. Financial support for this study was provided by AHRQ under contract # HHSA290200900012I, TO 4 .
Keywords
- Assessment
- Audiovisual materials
- Clear communication
- Educational materials
- Health literacy
- Instrument development
- Measurement
- Patient education
- Plain language
- Readability
ASJC Scopus subject areas
- General Medicine