Do End-of-Rotation Evaluations Adequately Assess Readiness to Operate?

Samantha L. Ahle*, Mary Schuller, Michael J. Clark, Reed G. Williams, Gregory Wnuk, Jonathan P. Fryer, Brian C. George

*Corresponding author for this work

Research output: Contribution to journalArticle

Abstract

Purpose: Medical educators have developed no standard way to assess the operative performance of surgical residents. Most residency programs use end-of-rotation (EOR) evaluations for this purpose. Recently, some programs have implemented workplace-based "microassessment" tools that faculty use to immediately rate observed operative performance. The authors sought to determine (1) the degree to which EOR evaluations correspond to workplace-based microassessments and (2) which factors most influence EOR evaluations and directly observed workplace-based performance ratings and how the influence of those factors differs for each assessment method. Method: In 2017, the authors retrospectively analyzed EOR evaluations and immediate postoperative assessment ratings of surgical trainees from a university-based training program from the 2015-2016 academic year. A Bayesian multivariate mixed model was constructed to predict operative performance ratings for each type of assessment. Results: Ratings of operative performance from EOR evaluations vs workplace-based microassessment ratings had a Pearson correlation of 0.55. Postgraduate year (PGY) of training was the most important predictor of operative performance ratings on EOR evaluations: Model estimates ranged from 0.62 to 1.75 and increased with PGY. For workplace-based assessment, operative autonomy rating was the most important predictor of operative performance (coefficient = 0.74). Conclusions: EOR evaluations are perhaps most useful in assessing the ability of a resident to become a surgeon compared with other trainees in the same PGY of training. Workplace-based microassessments may be better for assessing a trainee's ability to perform specific procedures autonomously, thus perhaps providing more insight into a trainee's true readiness for operative independence.

Original languageEnglish (US)
Pages (from-to)1946-1952
Number of pages7
JournalAcademic Medicine
Volume94
Issue number12
DOIs
StatePublished - Dec 1 2019

ASJC Scopus subject areas

  • Education

Fingerprint Dive into the research topics of 'Do End-of-Rotation Evaluations Adequately Assess Readiness to Operate?'. Together they form a unique fingerprint.

  • Cite this

    Ahle, S. L., Schuller, M., Clark, M. J., Williams, R. G., Wnuk, G., Fryer, J. P., & George, B. C. (2019). Do End-of-Rotation Evaluations Adequately Assess Readiness to Operate? Academic Medicine, 94(12), 1946-1952. https://doi.org/10.1097/ACM.0000000000002936