What Is the Relationship between a Preclerkship Portfolio Review and Later Performance in Clerkships?

Celia Laird O'Brien*, John X Thomas, Marianne Green

*Corresponding author for this work

Research output: Contribution to journalReview article

Abstract

Purpose Medical educators struggle to find effective ways to assess essential competencies such as communication, professionalism, and teamwork. Portfolio-based assessment provides one method of addressing this problem by allowing faculty reviewers to judge performance, as based on a longitudinal record of student behavior. At the Feinberg School of Medicine, the portfolio system measures behavioral competence using multiple assessments collected over time. This study examines whether a preclerkship portfolio review is a valid method of identifying problematic student behavior affecting later performance in clerkships. Method The authors divided students into two groups based on a summative preclerkship portfolio review in 2014: students who had concerning behavior in one or more competencies and students progressing satisfactorily. They compared how students in these groups later performed on two clerkship outcomes as of October 2015: final grades in required clerkships, and performance on a clerkship clinical composite score. They used Mann-Whitney tests and multiple linear regression to examine the relationship between portfolio review results and clerkship outcomes. They used USMLE Step 1 to control for knowledge acquisition. Results Students with concerning behavior preclerkship received significantly lower clerkship grades than students progressing satisfactorily (P =.002). They also scored significantly lower on the clinical composite score (P <.001). Regression analysis indicated concerning behavior was associated with lower clinical composite scores, even after controlling for knowledge acquisition. Conclusions The results show a preclerkship portfolio review can identify behaviors that impact clerkship performance. A comprehensive portfolio system is a valid way to measure behavioral competencies.

Original languageEnglish (US)
Pages (from-to)113-118
Number of pages6
JournalAcademic Medicine
Volume93
Issue number1
DOIs
StatePublished - Jan 1 2018

Fingerprint

performance
student
knowledge acquisition
teamwork
regression analysis
Group
medicine
educator
regression
communication
school

ASJC Scopus subject areas

  • Education

Cite this

@article{88af1fc549ce457f8863dbc772b31cd6,
title = "What Is the Relationship between a Preclerkship Portfolio Review and Later Performance in Clerkships?",
abstract = "Purpose Medical educators struggle to find effective ways to assess essential competencies such as communication, professionalism, and teamwork. Portfolio-based assessment provides one method of addressing this problem by allowing faculty reviewers to judge performance, as based on a longitudinal record of student behavior. At the Feinberg School of Medicine, the portfolio system measures behavioral competence using multiple assessments collected over time. This study examines whether a preclerkship portfolio review is a valid method of identifying problematic student behavior affecting later performance in clerkships. Method The authors divided students into two groups based on a summative preclerkship portfolio review in 2014: students who had concerning behavior in one or more competencies and students progressing satisfactorily. They compared how students in these groups later performed on two clerkship outcomes as of October 2015: final grades in required clerkships, and performance on a clerkship clinical composite score. They used Mann-Whitney tests and multiple linear regression to examine the relationship between portfolio review results and clerkship outcomes. They used USMLE Step 1 to control for knowledge acquisition. Results Students with concerning behavior preclerkship received significantly lower clerkship grades than students progressing satisfactorily (P =.002). They also scored significantly lower on the clinical composite score (P <.001). Regression analysis indicated concerning behavior was associated with lower clinical composite scores, even after controlling for knowledge acquisition. Conclusions The results show a preclerkship portfolio review can identify behaviors that impact clerkship performance. A comprehensive portfolio system is a valid way to measure behavioral competencies.",
author = "O'Brien, {Celia Laird} and Thomas, {John X} and Marianne Green",
year = "2018",
month = "1",
day = "1",
doi = "10.1097/ACM.0000000000001771",
language = "English (US)",
volume = "93",
pages = "113--118",
journal = "Academic Medicine",
issn = "1040-2446",
publisher = "Lippincott Williams and Wilkins",
number = "1",

}

TY - JOUR

T1 - What Is the Relationship between a Preclerkship Portfolio Review and Later Performance in Clerkships?

AU - O'Brien, Celia Laird

AU - Thomas, John X

AU - Green, Marianne

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Purpose Medical educators struggle to find effective ways to assess essential competencies such as communication, professionalism, and teamwork. Portfolio-based assessment provides one method of addressing this problem by allowing faculty reviewers to judge performance, as based on a longitudinal record of student behavior. At the Feinberg School of Medicine, the portfolio system measures behavioral competence using multiple assessments collected over time. This study examines whether a preclerkship portfolio review is a valid method of identifying problematic student behavior affecting later performance in clerkships. Method The authors divided students into two groups based on a summative preclerkship portfolio review in 2014: students who had concerning behavior in one or more competencies and students progressing satisfactorily. They compared how students in these groups later performed on two clerkship outcomes as of October 2015: final grades in required clerkships, and performance on a clerkship clinical composite score. They used Mann-Whitney tests and multiple linear regression to examine the relationship between portfolio review results and clerkship outcomes. They used USMLE Step 1 to control for knowledge acquisition. Results Students with concerning behavior preclerkship received significantly lower clerkship grades than students progressing satisfactorily (P =.002). They also scored significantly lower on the clinical composite score (P <.001). Regression analysis indicated concerning behavior was associated with lower clinical composite scores, even after controlling for knowledge acquisition. Conclusions The results show a preclerkship portfolio review can identify behaviors that impact clerkship performance. A comprehensive portfolio system is a valid way to measure behavioral competencies.

AB - Purpose Medical educators struggle to find effective ways to assess essential competencies such as communication, professionalism, and teamwork. Portfolio-based assessment provides one method of addressing this problem by allowing faculty reviewers to judge performance, as based on a longitudinal record of student behavior. At the Feinberg School of Medicine, the portfolio system measures behavioral competence using multiple assessments collected over time. This study examines whether a preclerkship portfolio review is a valid method of identifying problematic student behavior affecting later performance in clerkships. Method The authors divided students into two groups based on a summative preclerkship portfolio review in 2014: students who had concerning behavior in one or more competencies and students progressing satisfactorily. They compared how students in these groups later performed on two clerkship outcomes as of October 2015: final grades in required clerkships, and performance on a clerkship clinical composite score. They used Mann-Whitney tests and multiple linear regression to examine the relationship between portfolio review results and clerkship outcomes. They used USMLE Step 1 to control for knowledge acquisition. Results Students with concerning behavior preclerkship received significantly lower clerkship grades than students progressing satisfactorily (P =.002). They also scored significantly lower on the clinical composite score (P <.001). Regression analysis indicated concerning behavior was associated with lower clinical composite scores, even after controlling for knowledge acquisition. Conclusions The results show a preclerkship portfolio review can identify behaviors that impact clerkship performance. A comprehensive portfolio system is a valid way to measure behavioral competencies.

UR - http://www.scopus.com/inward/record.url?scp=85021183130&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85021183130&partnerID=8YFLogxK

U2 - 10.1097/ACM.0000000000001771

DO - 10.1097/ACM.0000000000001771

M3 - Review article

C2 - 28640026

AN - SCOPUS:85021183130

VL - 93

SP - 113

EP - 118

JO - Academic Medicine

JF - Academic Medicine

SN - 1040-2446

IS - 1

ER -