Residents' Procedural Experience Does Not Ensure Competence: A Research Synthesis

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

BACKGROUND: Many medical certifying bodies require that a minimum number of clinical procedures be completed during residency training to obtain board eligibility. However, little is known about the relationship between the number of procedures residents perform and their clinical competence.

OBJECTIVE: This study evaluated associations between residents' medical procedure skills measured in a simulation laboratory and self-reported procedure experience and year of training.

METHODS: This research synthesis extracted and summarized data from multiple cohorts of internal medicine, emergency medicine, anesthesiology, and neurology resident physicians who performed simulated clinical procedures. The procedures were central venous catheter insertion, lumbar puncture, paracentesis, and thoracentesis. We compared residents' baseline simulated performance to their self-reported procedure experience using data from 7 research reports written by Northwestern University investigators between 2006 and 2016. We also evaluated how performance differed by postgraduate year (PGY).

RESULTS: A total of 588 simulated procedures were performed during the study period. We found significant associations between passing the skills examinations and higher number of self-reported procedures performed (P = .011) and higher PGY (P < .001). However, performance for all procedures was poor, as only 10% of residents passed the assessments with a mean of 48% of checklist items correct (SD = 24.2). The association between passing the skills examination and year of training was mostly due to differences between PGY-1 and subsequent years of training.

CONCLUSIONS: Despite positive associations between self-reported experience and simulated procedure performance, overall performance was poor. Residents' clinical experience is not a proxy for skill.

Original languageEnglish (US)
Pages (from-to)201-208
Number of pages8
JournalJournal of graduate medical education
Volume9
Issue number2
DOIs
StatePublished - Apr 1 2017

Fingerprint

Mental Competency
Research
Ensure
Paracentesis
Clinical Competence
Spinal Puncture
Anesthesiology
Central Venous Catheters
Emergency Medicine
Proxy
Neurology
Internship and Residency
Internal Medicine
Checklist
Research Personnel
Physicians

ASJC Scopus subject areas

  • Medicine(all)

Cite this

@article{70dd593d5c98457b9850d8301579b9d9,
title = "Residents' Procedural Experience Does Not Ensure Competence: A Research Synthesis",
abstract = "BACKGROUND: Many medical certifying bodies require that a minimum number of clinical procedures be completed during residency training to obtain board eligibility. However, little is known about the relationship between the number of procedures residents perform and their clinical competence.OBJECTIVE: This study evaluated associations between residents' medical procedure skills measured in a simulation laboratory and self-reported procedure experience and year of training.METHODS: This research synthesis extracted and summarized data from multiple cohorts of internal medicine, emergency medicine, anesthesiology, and neurology resident physicians who performed simulated clinical procedures. The procedures were central venous catheter insertion, lumbar puncture, paracentesis, and thoracentesis. We compared residents' baseline simulated performance to their self-reported procedure experience using data from 7 research reports written by Northwestern University investigators between 2006 and 2016. We also evaluated how performance differed by postgraduate year (PGY).RESULTS: A total of 588 simulated procedures were performed during the study period. We found significant associations between passing the skills examinations and higher number of self-reported procedures performed (P = .011) and higher PGY (P < .001). However, performance for all procedures was poor, as only 10{\%} of residents passed the assessments with a mean of 48{\%} of checklist items correct (SD = 24.2). The association between passing the skills examination and year of training was mostly due to differences between PGY-1 and subsequent years of training.CONCLUSIONS: Despite positive associations between self-reported experience and simulated procedure performance, overall performance was poor. Residents' clinical experience is not a proxy for skill.",
author = "Barsuk, {Jeffrey H.} and Cohen, {Elaine R.} and Joe Feinglass and McGaghie, {William C.} and Wayne, {Diane B.}",
year = "2017",
month = "4",
day = "1",
doi = "10.4300/JGME-D-16-00426.1",
language = "English (US)",
volume = "9",
pages = "201--208",
journal = "Journal of graduate medical education",
issn = "1949-8349",
publisher = "University of Finance and Management",
number = "2",

}

TY - JOUR

T1 - Residents' Procedural Experience Does Not Ensure Competence

T2 - A Research Synthesis

AU - Barsuk, Jeffrey H.

AU - Cohen, Elaine R.

AU - Feinglass, Joe

AU - McGaghie, William C.

AU - Wayne, Diane B.

PY - 2017/4/1

Y1 - 2017/4/1

N2 - BACKGROUND: Many medical certifying bodies require that a minimum number of clinical procedures be completed during residency training to obtain board eligibility. However, little is known about the relationship between the number of procedures residents perform and their clinical competence.OBJECTIVE: This study evaluated associations between residents' medical procedure skills measured in a simulation laboratory and self-reported procedure experience and year of training.METHODS: This research synthesis extracted and summarized data from multiple cohorts of internal medicine, emergency medicine, anesthesiology, and neurology resident physicians who performed simulated clinical procedures. The procedures were central venous catheter insertion, lumbar puncture, paracentesis, and thoracentesis. We compared residents' baseline simulated performance to their self-reported procedure experience using data from 7 research reports written by Northwestern University investigators between 2006 and 2016. We also evaluated how performance differed by postgraduate year (PGY).RESULTS: A total of 588 simulated procedures were performed during the study period. We found significant associations between passing the skills examinations and higher number of self-reported procedures performed (P = .011) and higher PGY (P < .001). However, performance for all procedures was poor, as only 10% of residents passed the assessments with a mean of 48% of checklist items correct (SD = 24.2). The association between passing the skills examination and year of training was mostly due to differences between PGY-1 and subsequent years of training.CONCLUSIONS: Despite positive associations between self-reported experience and simulated procedure performance, overall performance was poor. Residents' clinical experience is not a proxy for skill.

AB - BACKGROUND: Many medical certifying bodies require that a minimum number of clinical procedures be completed during residency training to obtain board eligibility. However, little is known about the relationship between the number of procedures residents perform and their clinical competence.OBJECTIVE: This study evaluated associations between residents' medical procedure skills measured in a simulation laboratory and self-reported procedure experience and year of training.METHODS: This research synthesis extracted and summarized data from multiple cohorts of internal medicine, emergency medicine, anesthesiology, and neurology resident physicians who performed simulated clinical procedures. The procedures were central venous catheter insertion, lumbar puncture, paracentesis, and thoracentesis. We compared residents' baseline simulated performance to their self-reported procedure experience using data from 7 research reports written by Northwestern University investigators between 2006 and 2016. We also evaluated how performance differed by postgraduate year (PGY).RESULTS: A total of 588 simulated procedures were performed during the study period. We found significant associations between passing the skills examinations and higher number of self-reported procedures performed (P = .011) and higher PGY (P < .001). However, performance for all procedures was poor, as only 10% of residents passed the assessments with a mean of 48% of checklist items correct (SD = 24.2). The association between passing the skills examination and year of training was mostly due to differences between PGY-1 and subsequent years of training.CONCLUSIONS: Despite positive associations between self-reported experience and simulated procedure performance, overall performance was poor. Residents' clinical experience is not a proxy for skill.

UR - http://www.scopus.com/inward/record.url?scp=85039872365&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85039872365&partnerID=8YFLogxK

U2 - 10.4300/JGME-D-16-00426.1

DO - 10.4300/JGME-D-16-00426.1

M3 - Article

C2 - 28439354

AN - SCOPUS:85039872365

VL - 9

SP - 201

EP - 208

JO - Journal of graduate medical education

JF - Journal of graduate medical education

SN - 1949-8349

IS - 2

ER -