Feasibility and Outcomes of Implementing a Portfolio Assessment System Alongside a Traditional Grading System

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Purpose Portfolios are a powerful tool to collect and evaluate evidence of medical students' competence across time. However, comprehensive portfolio assessment systems that are implemented alongside traditional graded curricula at medical schools in the United States have not been described in the literature. This study describes the development and implementation of a longitudinal competency-based electronic portfolio system alongside a graded curriculum at a relatively large U.S. medical school. Method In 2009, the authors developed a portfolio system that served as a repository for all student assessments organized by competency domain. Five competencies were selected for a preclerkship summative portfolio review. Students submitted reflections on their performance. In 2014, four clinical faculty members participated in standard-setting activities and used expert judgment and holistic review to rate students' competency achievement as "progressing toward competence," "progressing toward competence with some concern," or "progressing toward competence pending remediation." Follow-up surveys measured students' and faculty members' perceptions of the process. Results Faculty evaluated 156 portfolios and showed high levels of agreement in their ratings. The majority of students achieved the "progressing toward competence" benchmark in all competency areas. However, 31 students received at least one concerning rating, which was not reflected in their course grades. Students' perceptions of the system's ability to foster self-assessment were mixed. Conclusions The portfolio review process allowed faculty to identify students with a concerning rating in a behavioral competency who would not have been identified in a traditional grading system. Identification of these students allows for intervention and early remediation.

Original languageEnglish (US)
Pages (from-to)1554-1560
Number of pages7
JournalAcademic Medicine
Volume91
Issue number11
DOIs
StatePublished - Nov 1 2016

Fingerprint

grading
Students
Mental Competency
student
rating
Medical Schools
Curriculum
Benchmarking
curriculum
Aptitude
self-assessment
Medical Students
school
medical student
electronics
expert
ability

ASJC Scopus subject areas

  • Medicine(all)
  • Education

Cite this

@article{cf4a9edd00bd41d2adac66f3c4823b5c,
title = "Feasibility and Outcomes of Implementing a Portfolio Assessment System Alongside a Traditional Grading System",
abstract = "Purpose Portfolios are a powerful tool to collect and evaluate evidence of medical students' competence across time. However, comprehensive portfolio assessment systems that are implemented alongside traditional graded curricula at medical schools in the United States have not been described in the literature. This study describes the development and implementation of a longitudinal competency-based electronic portfolio system alongside a graded curriculum at a relatively large U.S. medical school. Method In 2009, the authors developed a portfolio system that served as a repository for all student assessments organized by competency domain. Five competencies were selected for a preclerkship summative portfolio review. Students submitted reflections on their performance. In 2014, four clinical faculty members participated in standard-setting activities and used expert judgment and holistic review to rate students' competency achievement as {"}progressing toward competence,{"} {"}progressing toward competence with some concern,{"} or {"}progressing toward competence pending remediation.{"} Follow-up surveys measured students' and faculty members' perceptions of the process. Results Faculty evaluated 156 portfolios and showed high levels of agreement in their ratings. The majority of students achieved the {"}progressing toward competence{"} benchmark in all competency areas. However, 31 students received at least one concerning rating, which was not reflected in their course grades. Students' perceptions of the system's ability to foster self-assessment were mixed. Conclusions The portfolio review process allowed faculty to identify students with a concerning rating in a behavioral competency who would not have been identified in a traditional grading system. Identification of these students allows for intervention and early remediation.",
author = "O'Brien, {Celia Laird} and Sanguino, {Sandra Maria} and Thomas, {John X} and Marianne Green",
year = "2016",
month = "11",
day = "1",
doi = "10.1097/ACM.0000000000001168",
language = "English (US)",
volume = "91",
pages = "1554--1560",
journal = "Academic Medicine",
issn = "1040-2446",
publisher = "Lippincott Williams and Wilkins",
number = "11",

}

TY - JOUR

T1 - Feasibility and Outcomes of Implementing a Portfolio Assessment System Alongside a Traditional Grading System

AU - O'Brien, Celia Laird

AU - Sanguino, Sandra Maria

AU - Thomas, John X

AU - Green, Marianne

PY - 2016/11/1

Y1 - 2016/11/1

N2 - Purpose Portfolios are a powerful tool to collect and evaluate evidence of medical students' competence across time. However, comprehensive portfolio assessment systems that are implemented alongside traditional graded curricula at medical schools in the United States have not been described in the literature. This study describes the development and implementation of a longitudinal competency-based electronic portfolio system alongside a graded curriculum at a relatively large U.S. medical school. Method In 2009, the authors developed a portfolio system that served as a repository for all student assessments organized by competency domain. Five competencies were selected for a preclerkship summative portfolio review. Students submitted reflections on their performance. In 2014, four clinical faculty members participated in standard-setting activities and used expert judgment and holistic review to rate students' competency achievement as "progressing toward competence," "progressing toward competence with some concern," or "progressing toward competence pending remediation." Follow-up surveys measured students' and faculty members' perceptions of the process. Results Faculty evaluated 156 portfolios and showed high levels of agreement in their ratings. The majority of students achieved the "progressing toward competence" benchmark in all competency areas. However, 31 students received at least one concerning rating, which was not reflected in their course grades. Students' perceptions of the system's ability to foster self-assessment were mixed. Conclusions The portfolio review process allowed faculty to identify students with a concerning rating in a behavioral competency who would not have been identified in a traditional grading system. Identification of these students allows for intervention and early remediation.

AB - Purpose Portfolios are a powerful tool to collect and evaluate evidence of medical students' competence across time. However, comprehensive portfolio assessment systems that are implemented alongside traditional graded curricula at medical schools in the United States have not been described in the literature. This study describes the development and implementation of a longitudinal competency-based electronic portfolio system alongside a graded curriculum at a relatively large U.S. medical school. Method In 2009, the authors developed a portfolio system that served as a repository for all student assessments organized by competency domain. Five competencies were selected for a preclerkship summative portfolio review. Students submitted reflections on their performance. In 2014, four clinical faculty members participated in standard-setting activities and used expert judgment and holistic review to rate students' competency achievement as "progressing toward competence," "progressing toward competence with some concern," or "progressing toward competence pending remediation." Follow-up surveys measured students' and faculty members' perceptions of the process. Results Faculty evaluated 156 portfolios and showed high levels of agreement in their ratings. The majority of students achieved the "progressing toward competence" benchmark in all competency areas. However, 31 students received at least one concerning rating, which was not reflected in their course grades. Students' perceptions of the system's ability to foster self-assessment were mixed. Conclusions The portfolio review process allowed faculty to identify students with a concerning rating in a behavioral competency who would not have been identified in a traditional grading system. Identification of these students allows for intervention and early remediation.

UR - http://www.scopus.com/inward/record.url?scp=84961942934&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84961942934&partnerID=8YFLogxK

U2 - 10.1097/ACM.0000000000001168

DO - 10.1097/ACM.0000000000001168

M3 - Article

VL - 91

SP - 1554

EP - 1560

JO - Academic Medicine

JF - Academic Medicine

SN - 1040-2446

IS - 11

ER -