An Analysis of Written and Numeric Scores in End-of-Rotation Forms from Three Residency Programs

Lauren M. Anderson*, Kathleen Rowland, Deborah Edberg, Katherine M. Wright, Yoon Soo Park, Ara Tekian

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Introduction: End-of-Rotation Forms (EORFs) assess resident progress in graduate medical education and are a major component of Clinical Competency Committee (CCC) discussion. Single-institution studies suggest EORFs can detect deficiencies, but both grades and comments skew positive. In this study, we sought to determine whether the EORFs from three programs, including multiple specialties and institutions, produced useful information for residents, program directors, and CCCs. Methods: Evaluations from three programs were included (Program 1, Institution A, Internal Medicine: n = 38; Program 2, Institution A, Anesthesia: n = 9; Program 3, Institution B, Anesthesia: n = 11). Two independent researchers coded each written comment for relevance (specificity and actionability) and orientation (praise or critical) using a standardized rubric. Numeric scores were analyzed using descriptive statistics. Results: 4869 evaluations were collected from the programs. Of the 77,434 discrete numeric scores, 691 (0.89%) were considered “below expected level.” 71.2% (2683/3767) of the total written comments were scored as irrelevant, while 3217 (85.4%) of total comments were scored positive and 550 (14.6%) were critical. When combined, 63.2% (n = 2379) of comments were scored positive and irrelevant while 6.5% (n = 246) were scored critical and relevant. Discussion: <1% of comments indicated below average performance; >70% of comments scored irrelevant. Critical, relevant comments were least frequently observed, consistent across all 3 programs. The low rate of constructive feedback and the high rate of irrelevant comments are inadequate for a CCC to make informed decisions. The consistency of these findings across programs, specialties, and institutions suggests both local and systemic changes should be considered.

Original languageEnglish (US)
Pages (from-to)497-506
Number of pages10
JournalPerspectives on Medical Education
Issue number1
StatePublished - 2023

ASJC Scopus subject areas

  • Education


Dive into the research topics of 'An Analysis of Written and Numeric Scores in End-of-Rotation Forms from Three Residency Programs'. Together they form a unique fingerprint.

Cite this