Validity and reliability of a sensor-enabled intubation trainer: A focus on patient-centered data

Background: Prior work using simulation for assessing intubation skills has largely focused on the use of observer-generated performance measures in the form of checklists and global ratings scales. Purpose: The purpose of our work was to investigate whether patient-centered simulation data could be used to quantify learner's performance during direct laryngoscopy. Methods: We designed a pretest/posttest prospective intervention study of residents' (n = 25) intubation skills. Results: When assessing validity, all of the patient-centered simulation variables showed significant correlations with the previously validated observer-generated performance measures (r = 0.331-0.463, P ≤ 0.001). When assessing reliability, there were significant correlations between all of the sensor variables, confirming moderate to high inter-item reliability (r = 0.259-0.794, P ≤ 0.05). The observer-generated performance measures showed significant improvement in use of the Macintosh blade (T1 = 2.10/5.00, T2 = 3.64/5.00, P = 0.001). However, this was not the case for the Miller blade (T1 = 1.30/5.00, T2 = 1.75/5.00, P = 0.119). Overall, the patient-centered simulation variables provided a high level of detail regarding performance improvement areas. Conclusion: This study presents a multilevel analysis of sensor-generated simulation data. As the sensors provide sound, formative data regarding patient contact, the outputs may be used for specific criterion measures and detailed performance feedback.