Comparison of content on the American board of internal medicine maintenance of certification examination with conditions seen in practice by general internists

Bradley Gray*, Jonathan Vandergrift, Rebecca S. Lipner, Marianne Green

*Corresponding author for this work

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

IMPORTANCE Success on the internal medicine (IM) examination is a central requirement of the American Board of Internal Medicine's (ABIM's) Maintenance of Certification program (MOC). Therefore, it is important to understand the degree to which this examination reflects conditions seen in practice, one dimension of content validity, which focuses on the match between content in the discipline and the topics on the examination questions. OBJECTIVE To assess whether the frequency of questions on IM-MOC examinations were concordant with the frequency of conditions seen in practice. DESIGN, SETTING, AND PARTICIPANTS The 2010-2013 IM-MOC examinationswere used to calculate the percentage of questions for 186 medical condition categories from the examination blueprint, which balances examination content by considering importance and frequency of conditions seen in practice. Nationally representative estimates of conditions seen in practice by general internists were estimated from the primary diagnosis for 13 832 office visits (2010-2013 National Ambulatory Medical Care Surveys) and 108 472 hospital stays (2010 National Hospital Discharge Survey). EXPOSURES Prevalence of conditions included on the IM-MOC examination questions. MAIN OUTCOMES AND MEASURES The outcome measurewas the concordance between the percentages of IM-MOC examination questions and the percentages of conditions seen in practice during either office visits or hospital stays for each of 186 condition categories (eg, diabetes mellitus, ischemic heart disease, liver disease). The concordance thresholds were 0.5 SD of the weighted mean percentages of the applicable 186 conditions seen in practice (0.74%for office visits; 0.51%for hospital stays). If the absolute differences between the percentages of examination questions and the percentages of office visit conditions or hospital stay conditions seen were less than the applicable concordance threshold, then the condition category was judged to be concordant. RESULTS During the 2010-2013 IM-MOC examination periods, 3600 questions (180 questions per examination form) were administered and 3461 questions (96.1%) were mapped into the 186 study conditions (mean, 18.6 questions per condition). Comparison of the percentages of 186 categories of medical conditions seen in 13 832 office visits and 108 472 hospital stays with the percentages of 3461 questions on IM-MOC examinations revealed that 2389 examination questions (69.0%; 95%CI, 67.5%-70.6%involving 158 conditions) were categorized as concordant. For concordance between questions and office visits only, 2010 questions (58.08%; 95%CI, 56.43%-59.72%of all examination questions) involving 145 conditions were categorized as concordant. For concordance between questions and hospital stays only, 1456 questions (42.07%; 95%CI, 40.42%-43.71% of all examination questions) involving 122 conditions were categorized as concordant. CONCLUSIONS AND RELEVANCE Among questions on IM-MOC examinations from 2010-2013, 69%were concordant with conditions seen in general internal medicine practices, although some areas of discordance were identified.

Original languageEnglish (US)
Pages (from-to)2317-2324
Number of pages8
JournalJAMA - Journal of the American Medical Association
Volume317
Issue number22
DOIs
StatePublished - Jun 13 2017

Fingerprint

Certification
Internal Medicine
General Practice
Office Visits
Maintenance
Length of Stay
Health Care Surveys
Myocardial Ischemia
Liver Diseases
Diabetes Mellitus

ASJC Scopus subject areas

  • Medicine(all)

Cite this

@article{eae109dedef64e4288267e95611d8da1,
title = "Comparison of content on the American board of internal medicine maintenance of certification examination with conditions seen in practice by general internists",
abstract = "IMPORTANCE Success on the internal medicine (IM) examination is a central requirement of the American Board of Internal Medicine's (ABIM's) Maintenance of Certification program (MOC). Therefore, it is important to understand the degree to which this examination reflects conditions seen in practice, one dimension of content validity, which focuses on the match between content in the discipline and the topics on the examination questions. OBJECTIVE To assess whether the frequency of questions on IM-MOC examinations were concordant with the frequency of conditions seen in practice. DESIGN, SETTING, AND PARTICIPANTS The 2010-2013 IM-MOC examinationswere used to calculate the percentage of questions for 186 medical condition categories from the examination blueprint, which balances examination content by considering importance and frequency of conditions seen in practice. Nationally representative estimates of conditions seen in practice by general internists were estimated from the primary diagnosis for 13 832 office visits (2010-2013 National Ambulatory Medical Care Surveys) and 108 472 hospital stays (2010 National Hospital Discharge Survey). EXPOSURES Prevalence of conditions included on the IM-MOC examination questions. MAIN OUTCOMES AND MEASURES The outcome measurewas the concordance between the percentages of IM-MOC examination questions and the percentages of conditions seen in practice during either office visits or hospital stays for each of 186 condition categories (eg, diabetes mellitus, ischemic heart disease, liver disease). The concordance thresholds were 0.5 SD of the weighted mean percentages of the applicable 186 conditions seen in practice (0.74{\%}for office visits; 0.51{\%}for hospital stays). If the absolute differences between the percentages of examination questions and the percentages of office visit conditions or hospital stay conditions seen were less than the applicable concordance threshold, then the condition category was judged to be concordant. RESULTS During the 2010-2013 IM-MOC examination periods, 3600 questions (180 questions per examination form) were administered and 3461 questions (96.1{\%}) were mapped into the 186 study conditions (mean, 18.6 questions per condition). Comparison of the percentages of 186 categories of medical conditions seen in 13 832 office visits and 108 472 hospital stays with the percentages of 3461 questions on IM-MOC examinations revealed that 2389 examination questions (69.0{\%}; 95{\%}CI, 67.5{\%}-70.6{\%}involving 158 conditions) were categorized as concordant. For concordance between questions and office visits only, 2010 questions (58.08{\%}; 95{\%}CI, 56.43{\%}-59.72{\%}of all examination questions) involving 145 conditions were categorized as concordant. For concordance between questions and hospital stays only, 1456 questions (42.07{\%}; 95{\%}CI, 40.42{\%}-43.71{\%} of all examination questions) involving 122 conditions were categorized as concordant. CONCLUSIONS AND RELEVANCE Among questions on IM-MOC examinations from 2010-2013, 69{\%}were concordant with conditions seen in general internal medicine practices, although some areas of discordance were identified.",
author = "Bradley Gray and Jonathan Vandergrift and Lipner, {Rebecca S.} and Marianne Green",
year = "2017",
month = "6",
day = "13",
doi = "10.1001/jama.2017.6853",
language = "English (US)",
volume = "317",
pages = "2317--2324",
journal = "JAMA - Journal of the American Medical Association",
issn = "0098-7484",
publisher = "American Medical Association",
number = "22",

}

Comparison of content on the American board of internal medicine maintenance of certification examination with conditions seen in practice by general internists. / Gray, Bradley; Vandergrift, Jonathan; Lipner, Rebecca S.; Green, Marianne.

In: JAMA - Journal of the American Medical Association, Vol. 317, No. 22, 13.06.2017, p. 2317-2324.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Comparison of content on the American board of internal medicine maintenance of certification examination with conditions seen in practice by general internists

AU - Gray, Bradley

AU - Vandergrift, Jonathan

AU - Lipner, Rebecca S.

AU - Green, Marianne

PY - 2017/6/13

Y1 - 2017/6/13

N2 - IMPORTANCE Success on the internal medicine (IM) examination is a central requirement of the American Board of Internal Medicine's (ABIM's) Maintenance of Certification program (MOC). Therefore, it is important to understand the degree to which this examination reflects conditions seen in practice, one dimension of content validity, which focuses on the match between content in the discipline and the topics on the examination questions. OBJECTIVE To assess whether the frequency of questions on IM-MOC examinations were concordant with the frequency of conditions seen in practice. DESIGN, SETTING, AND PARTICIPANTS The 2010-2013 IM-MOC examinationswere used to calculate the percentage of questions for 186 medical condition categories from the examination blueprint, which balances examination content by considering importance and frequency of conditions seen in practice. Nationally representative estimates of conditions seen in practice by general internists were estimated from the primary diagnosis for 13 832 office visits (2010-2013 National Ambulatory Medical Care Surveys) and 108 472 hospital stays (2010 National Hospital Discharge Survey). EXPOSURES Prevalence of conditions included on the IM-MOC examination questions. MAIN OUTCOMES AND MEASURES The outcome measurewas the concordance between the percentages of IM-MOC examination questions and the percentages of conditions seen in practice during either office visits or hospital stays for each of 186 condition categories (eg, diabetes mellitus, ischemic heart disease, liver disease). The concordance thresholds were 0.5 SD of the weighted mean percentages of the applicable 186 conditions seen in practice (0.74%for office visits; 0.51%for hospital stays). If the absolute differences between the percentages of examination questions and the percentages of office visit conditions or hospital stay conditions seen were less than the applicable concordance threshold, then the condition category was judged to be concordant. RESULTS During the 2010-2013 IM-MOC examination periods, 3600 questions (180 questions per examination form) were administered and 3461 questions (96.1%) were mapped into the 186 study conditions (mean, 18.6 questions per condition). Comparison of the percentages of 186 categories of medical conditions seen in 13 832 office visits and 108 472 hospital stays with the percentages of 3461 questions on IM-MOC examinations revealed that 2389 examination questions (69.0%; 95%CI, 67.5%-70.6%involving 158 conditions) were categorized as concordant. For concordance between questions and office visits only, 2010 questions (58.08%; 95%CI, 56.43%-59.72%of all examination questions) involving 145 conditions were categorized as concordant. For concordance between questions and hospital stays only, 1456 questions (42.07%; 95%CI, 40.42%-43.71% of all examination questions) involving 122 conditions were categorized as concordant. CONCLUSIONS AND RELEVANCE Among questions on IM-MOC examinations from 2010-2013, 69%were concordant with conditions seen in general internal medicine practices, although some areas of discordance were identified.

AB - IMPORTANCE Success on the internal medicine (IM) examination is a central requirement of the American Board of Internal Medicine's (ABIM's) Maintenance of Certification program (MOC). Therefore, it is important to understand the degree to which this examination reflects conditions seen in practice, one dimension of content validity, which focuses on the match between content in the discipline and the topics on the examination questions. OBJECTIVE To assess whether the frequency of questions on IM-MOC examinations were concordant with the frequency of conditions seen in practice. DESIGN, SETTING, AND PARTICIPANTS The 2010-2013 IM-MOC examinationswere used to calculate the percentage of questions for 186 medical condition categories from the examination blueprint, which balances examination content by considering importance and frequency of conditions seen in practice. Nationally representative estimates of conditions seen in practice by general internists were estimated from the primary diagnosis for 13 832 office visits (2010-2013 National Ambulatory Medical Care Surveys) and 108 472 hospital stays (2010 National Hospital Discharge Survey). EXPOSURES Prevalence of conditions included on the IM-MOC examination questions. MAIN OUTCOMES AND MEASURES The outcome measurewas the concordance between the percentages of IM-MOC examination questions and the percentages of conditions seen in practice during either office visits or hospital stays for each of 186 condition categories (eg, diabetes mellitus, ischemic heart disease, liver disease). The concordance thresholds were 0.5 SD of the weighted mean percentages of the applicable 186 conditions seen in practice (0.74%for office visits; 0.51%for hospital stays). If the absolute differences between the percentages of examination questions and the percentages of office visit conditions or hospital stay conditions seen were less than the applicable concordance threshold, then the condition category was judged to be concordant. RESULTS During the 2010-2013 IM-MOC examination periods, 3600 questions (180 questions per examination form) were administered and 3461 questions (96.1%) were mapped into the 186 study conditions (mean, 18.6 questions per condition). Comparison of the percentages of 186 categories of medical conditions seen in 13 832 office visits and 108 472 hospital stays with the percentages of 3461 questions on IM-MOC examinations revealed that 2389 examination questions (69.0%; 95%CI, 67.5%-70.6%involving 158 conditions) were categorized as concordant. For concordance between questions and office visits only, 2010 questions (58.08%; 95%CI, 56.43%-59.72%of all examination questions) involving 145 conditions were categorized as concordant. For concordance between questions and hospital stays only, 1456 questions (42.07%; 95%CI, 40.42%-43.71% of all examination questions) involving 122 conditions were categorized as concordant. CONCLUSIONS AND RELEVANCE Among questions on IM-MOC examinations from 2010-2013, 69%were concordant with conditions seen in general internal medicine practices, although some areas of discordance were identified.

UR - http://www.scopus.com/inward/record.url?scp=85020899319&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020899319&partnerID=8YFLogxK

U2 - 10.1001/jama.2017.6853

DO - 10.1001/jama.2017.6853

M3 - Article

VL - 317

SP - 2317

EP - 2324

JO - JAMA - Journal of the American Medical Association

JF - JAMA - Journal of the American Medical Association

SN - 0098-7484

IS - 22

ER -