The Development and Validation of a Concise Instrument for Formative Assessment of Team Leader Performance during Simulated Pediatric Resuscitations

Lindsay D. Nadkarni*, Cindy G. Roskind, Marc A. Auerbach, Aaron W. Calhoun, Mark D. Adler, David O. Kessler

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

24 Scopus citations

Abstract

Aim The aim of this study was to assess the validity of a formative feedback instrument for leaders of simulated resuscitations. Methods This is a prospective validation study with a fully crossed (person × scenario × rater) study design. The Concise Assessment of Leader Management (CALM) instrument was designed by pediatric emergency medicine and graduate medical education experts to be used off the shelf to evaluate and provide formative feedback to resuscitation leaders. Four experts reviewed 16 videos of in situ simulated pediatric resuscitations and scored resuscitation leader performance using the CALM instrument. The videos consisted of 4 pediatric emergency department resuscitation teams each performing in 4 pediatric resuscitation scenarios (cardiac arrest, respiratory arrest, seizure, and sepsis). We report on content and internal structure (reliability) validity of the CALM instrument. Results Content validity was supported by the instrument development process that involved professional experience, expert consensus, focused literature review, and pilot testing. Internal structure validity (reliability) was supported by the generalizability analysis. The main component that contributed to score variability was the person (33%), meaning that individual leaders performed differently. The rater component had almost zero (0%) contribution to variance, which implies that raters were in agreement and argues for high interrater reliability. Conclusions These results provide initial evidence to support the validity of the CALM instrument as a reliable assessment instrument that can facilitate formative feedback to leaders of pediatric simulated resuscitations.

Original languageEnglish (US)
Pages (from-to)77-82
Number of pages6
JournalSimulation in Healthcare
Volume13
Issue number2
DOIs
StatePublished - Apr 1 2018

Keywords

  • Resuscitation
  • Simulation
  • Team leader

ASJC Scopus subject areas

  • Education
  • Epidemiology
  • Medicine (miscellaneous)
  • Modeling and Simulation

Fingerprint

Dive into the research topics of 'The Development and Validation of a Concise Instrument for Formative Assessment of Team Leader Performance during Simulated Pediatric Resuscitations'. Together they form a unique fingerprint.

Cite this