A comparison of approaches for mastery learning standard setting

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Purpose Defensible minimum passing standards (MPSs) must be used to evaluate learner performance outcomes in health professions education. In this study, the authors compared the results of traditional Angoff and Hofstee standard-setting exercises with the Mastery Angoff and Patient-Safety approaches for central venous catheter (CVC) insertion skills examinations. The authors also evaluated how these standards affected the historical performance of residents who participated in a simulation-based mastery learning (SBML) curriculum for CVC insertion skills. Method In April and May 2015, 12 physicians with expertise in CVC insertion set MPSs for previously published internal jugular (IJ) and subclavian (SC) CVC insertion checklists using Angoff, Hofstee, Mastery Angoff, and Patient-Safety approaches. The resulting MPSs were compared using historical performance of internal medicine and emergency medicine residents who participated in CVC insertion SBML. Results The MPSs were set as follows: Angoff: IJ 91% checklist items correct, SC 90%. Hofstee: IJ 88%, SC 90%. Mastery Angoff: IJ 98%, SC 98%. Patient-Safety: IJ 98%, SC 98%. Based on the historical performance of 143 residents assessed on IJ and SC insertion, applying the 98% MPS would result in additional practice and retesting of 55/123 residents (45%) who had previously passed the IJ examination and 36/130 residents (28%) who had passed the SC examination using the Angoff and Hofstee MPSs. Conclusions The Mastery Angoff and Patient-Safety standard-setting approaches resulted in higher CVC insertion SBML MPSs compared with traditional standard-setting methods. Further study should assess the impact of these more rigorous standards on patient outcomes.

Original languageEnglish (US)
Pages (from-to)1079-1084
Number of pages6
JournalAcademic Medicine
Volume93
Issue number7
DOIs
StatePublished - Jul 1 2018

Fingerprint

learning
resident
examination
simulation
performance
medicine
expertise
profession
physician
curriculum
health
education

ASJC Scopus subject areas

  • Education

Cite this

@article{a1968ed628894d7f8aeeb473ddea110c,
title = "A comparison of approaches for mastery learning standard setting",
abstract = "Purpose Defensible minimum passing standards (MPSs) must be used to evaluate learner performance outcomes in health professions education. In this study, the authors compared the results of traditional Angoff and Hofstee standard-setting exercises with the Mastery Angoff and Patient-Safety approaches for central venous catheter (CVC) insertion skills examinations. The authors also evaluated how these standards affected the historical performance of residents who participated in a simulation-based mastery learning (SBML) curriculum for CVC insertion skills. Method In April and May 2015, 12 physicians with expertise in CVC insertion set MPSs for previously published internal jugular (IJ) and subclavian (SC) CVC insertion checklists using Angoff, Hofstee, Mastery Angoff, and Patient-Safety approaches. The resulting MPSs were compared using historical performance of internal medicine and emergency medicine residents who participated in CVC insertion SBML. Results The MPSs were set as follows: Angoff: IJ 91{\%} checklist items correct, SC 90{\%}. Hofstee: IJ 88{\%}, SC 90{\%}. Mastery Angoff: IJ 98{\%}, SC 98{\%}. Patient-Safety: IJ 98{\%}, SC 98{\%}. Based on the historical performance of 143 residents assessed on IJ and SC insertion, applying the 98{\%} MPS would result in additional practice and retesting of 55/123 residents (45{\%}) who had previously passed the IJ examination and 36/130 residents (28{\%}) who had passed the SC examination using the Angoff and Hofstee MPSs. Conclusions The Mastery Angoff and Patient-Safety standard-setting approaches resulted in higher CVC insertion SBML MPSs compared with traditional standard-setting methods. Further study should assess the impact of these more rigorous standards on patient outcomes.",
author = "Barsuk, {Jeffrey H} and Cohen, {Elaine R.} and Wayne, {Diane Bronstein} and McGaghie, {William Craig} and Rachel Yudkowsky",
year = "2018",
month = "7",
day = "1",
doi = "10.1097/ACM.0000000000002182",
language = "English (US)",
volume = "93",
pages = "1079--1084",
journal = "Academic Medicine",
issn = "1040-2446",
publisher = "Lippincott Williams and Wilkins",
number = "7",

}

A comparison of approaches for mastery learning standard setting. / Barsuk, Jeffrey H; Cohen, Elaine R.; Wayne, Diane Bronstein; McGaghie, William Craig; Yudkowsky, Rachel.

In: Academic Medicine, Vol. 93, No. 7, 01.07.2018, p. 1079-1084.

Research output: Contribution to journalArticle

TY - JOUR

T1 - A comparison of approaches for mastery learning standard setting

AU - Barsuk, Jeffrey H

AU - Cohen, Elaine R.

AU - Wayne, Diane Bronstein

AU - McGaghie, William Craig

AU - Yudkowsky, Rachel

PY - 2018/7/1

Y1 - 2018/7/1

N2 - Purpose Defensible minimum passing standards (MPSs) must be used to evaluate learner performance outcomes in health professions education. In this study, the authors compared the results of traditional Angoff and Hofstee standard-setting exercises with the Mastery Angoff and Patient-Safety approaches for central venous catheter (CVC) insertion skills examinations. The authors also evaluated how these standards affected the historical performance of residents who participated in a simulation-based mastery learning (SBML) curriculum for CVC insertion skills. Method In April and May 2015, 12 physicians with expertise in CVC insertion set MPSs for previously published internal jugular (IJ) and subclavian (SC) CVC insertion checklists using Angoff, Hofstee, Mastery Angoff, and Patient-Safety approaches. The resulting MPSs were compared using historical performance of internal medicine and emergency medicine residents who participated in CVC insertion SBML. Results The MPSs were set as follows: Angoff: IJ 91% checklist items correct, SC 90%. Hofstee: IJ 88%, SC 90%. Mastery Angoff: IJ 98%, SC 98%. Patient-Safety: IJ 98%, SC 98%. Based on the historical performance of 143 residents assessed on IJ and SC insertion, applying the 98% MPS would result in additional practice and retesting of 55/123 residents (45%) who had previously passed the IJ examination and 36/130 residents (28%) who had passed the SC examination using the Angoff and Hofstee MPSs. Conclusions The Mastery Angoff and Patient-Safety standard-setting approaches resulted in higher CVC insertion SBML MPSs compared with traditional standard-setting methods. Further study should assess the impact of these more rigorous standards on patient outcomes.

AB - Purpose Defensible minimum passing standards (MPSs) must be used to evaluate learner performance outcomes in health professions education. In this study, the authors compared the results of traditional Angoff and Hofstee standard-setting exercises with the Mastery Angoff and Patient-Safety approaches for central venous catheter (CVC) insertion skills examinations. The authors also evaluated how these standards affected the historical performance of residents who participated in a simulation-based mastery learning (SBML) curriculum for CVC insertion skills. Method In April and May 2015, 12 physicians with expertise in CVC insertion set MPSs for previously published internal jugular (IJ) and subclavian (SC) CVC insertion checklists using Angoff, Hofstee, Mastery Angoff, and Patient-Safety approaches. The resulting MPSs were compared using historical performance of internal medicine and emergency medicine residents who participated in CVC insertion SBML. Results The MPSs were set as follows: Angoff: IJ 91% checklist items correct, SC 90%. Hofstee: IJ 88%, SC 90%. Mastery Angoff: IJ 98%, SC 98%. Patient-Safety: IJ 98%, SC 98%. Based on the historical performance of 143 residents assessed on IJ and SC insertion, applying the 98% MPS would result in additional practice and retesting of 55/123 residents (45%) who had previously passed the IJ examination and 36/130 residents (28%) who had passed the SC examination using the Angoff and Hofstee MPSs. Conclusions The Mastery Angoff and Patient-Safety standard-setting approaches resulted in higher CVC insertion SBML MPSs compared with traditional standard-setting methods. Further study should assess the impact of these more rigorous standards on patient outcomes.

UR - http://www.scopus.com/inward/record.url?scp=85049205371&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85049205371&partnerID=8YFLogxK

U2 - 10.1097/ACM.0000000000002182

DO - 10.1097/ACM.0000000000002182

M3 - Article

VL - 93

SP - 1079

EP - 1084

JO - Academic Medicine

JF - Academic Medicine

SN - 1040-2446

IS - 7

ER -