Surgery Clerkship Evaluations Are Insufficient for Clinical Skills Appraisal: The Value of a Medical Student Surgical Objective Structured Clinical Examination

Kathryn L. Butler*, David A. Hirsh, Emil R. Petrusa, D. Dante Yeh, Dana Stearns, David E. Sloane, Jeffrey A. Linder, Gaurab Basu, Lisa A. Thompson, Marc A. de Moya

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

7 Scopus citations

Abstract

Objective Optimal methods for medical student assessment in surgery remain elusive. Faculty- and housestaff-written evaluations constitute the chief means of student assessment in medical education. However, numerous studies show that this approach has poor specificity and a high degree of subjectivity. We hypothesized that an objective structured clinical examination (OSCE) in the surgery clerkship would provide additional data on student performance that would confirm or augment other measures of assessment. Design We retrospectively reviewed data from OSCEs, National Board of Medical Examiners shelf examinations, oral presentations, and written evaluations for 51 third-year Harvard Medical School students rotating in surgery at Massachusetts General Hospital from 2014 to 2015. We expressed correlations between numeric variables in Pearson coefficients, stratified differences between rater groups by one-way analysis of variance, and compared percentages with 2-sample t-tests. We examined commentary from both OSCE and clinical written evaluations through textual analysis and summarized these results in percentages. Results OSCE scores and clinical evaluation scores correlated poorly with each other, as well as with shelf examination scores and oral presentation grades. Textual analysis of clinical evaluation comments revealed a heavy emphasis on motivational factors and praise, whereas OSCE written comments focused on cognitive processes, patient management, and methods to improve performance. Conclusions In this single-center study, an OSCE provided clinical skills data that were not captured elsewhere in the surgery clerkship. Textual analysis of faculty evaluations reflected an emphasis on interpersonal skills, rather than appraisal of clinical acumen. These findings suggest complementary roles of faculty evaluations and OSCEs in medical student assessment.

Original languageEnglish (US)
Pages (from-to)286-294
Number of pages9
JournalJournal of Surgical Education
Volume74
Issue number2
DOIs
StatePublished - Mar 1 2017

Keywords

  • Interpersonal Skills and Communication
  • Medical Knowledge
  • Patient Care
  • Professionalism
  • clinical evaluations
  • medical student assessment
  • objective structured clinical skills examination
  • simulation
  • surgery clerkship
  • textual analysis

ASJC Scopus subject areas

  • Surgery
  • Education

Fingerprint Dive into the research topics of 'Surgery Clerkship Evaluations Are Insufficient for Clinical Skills Appraisal: The Value of a Medical Student Surgical Objective Structured Clinical Examination'. Together they form a unique fingerprint.

Cite this