Can skills coaches be used to assess resident performance in the skills laboratory?

Deborah M. Rooney*, Eric S. Hungness, Debra A. Darosa, Carla M. Pugh

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

Introduction: The purpose of this study was to compare faculty ratings between live versus video-recorded resident performances and faculty versus skills coaches' ratings of video-recorded resident performances. Methods: PGY1 residents were observed, video-recorded, and rated during a Verification of Proficiency examination on 4 stations (ie, suturing, laparotomy, central line, and cricothyroidotomy). One surgeon and 2 trained skills coaches independently rated each video-recorded performance (N = 25). The chi-square test was used to compare checklist ratings. Analysis of variance was used to compare global ratings. Intraclass correlations were used to evaluate inter-rater agreement. Results: There were no statistical differences in faculty checklist ratings for live versus video-recorded performances (P >.05), and we found a nearly perfect interrater agreement, intraclass correlation coefficient (ICC) = 0.99 (P <.001). When comparing faculty versus skills coaches' ratings on video-recorded performances, we found no differences for the global or checklist ratings. Inter-rater agreement was moderately high for the global ratings, ICC = 0.71 (P <. 0.01, 95% confidence interval 0.23-0.96), and nearly perfect for the checklist ratings, ICC = 0.99 (P <.001, 95% confidence interval 0.94-1.00). Conclusion: When assessing residents' performances, use of video-recorded performance ratings and skills coaches may be viable alternatives to live ratings performed by surgical faculty.

Original languageEnglish (US)
Pages (from-to)796-802
Number of pages7
JournalSurgery (United States)
Volume151
Issue number6
DOIs
StatePublished - Jun 1 2012

ASJC Scopus subject areas

  • Surgery

Fingerprint Dive into the research topics of 'Can skills coaches be used to assess resident performance in the skills laboratory?'. Together they form a unique fingerprint.

Cite this