Incorporating Physician Input into a Maintenance of Certification Examination: A Content Validity Tool

Paul A. Poniatowski, Jeremy W. Dugosh, Rebecca A. Baranowski*, Gerald Arnold, Rebecca S. Lipner, George W. Dec, Marianne M. Green

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Scopus citations

Abstract

Purpose As part of the American Board of Internal Medicine's (ABIM's) continuing effort to update its Maintenance of Certification (MOC) program, a content validity tool was used to conduct structured reviews of MOC exam blueprints (i.e., test specification tables) by the physician community. Results from the Cardiovascular Disease MOC blueprint review are presented to illustrate the process ABIM conducted for several internal medicine disciplines. Method Ratings of topic frequency and importance were collected from cardiologists in 2016 using a three-point scale (low, medium, high). The web-based survey instrument presented 188 blueprint topic descriptions, each combined with five patient-related tasks (e.g., diagnosis, treatment). Descriptive statistics and chi-square analysis were employed. Results Responses from 441 review participants were analyzed. Frequency and importance ratings were aggregated as a composite statistic representing clinical relevance, and exam assembly criteria were modified to select questions, or items, addressing clinically relevant content only. Specifically, ≥ 88% of exam items now address high-importance topics, including ≤ 15% on topics that are also low frequency; and ≤ 12% of exam items now address medium-importance topics, including ≤ 3% on topics that are also low frequency. The updated blueprint has been published for test takers and provides enhanced information on content that would and would not be tested in subsequent examinations. It is linked to more detailed feedback that examinees receive on items answered incorrectly. Conclusions The blueprint review garnered valuable feedback from the physician community and provided new evidence for the content validity of the Cardiovascular Disease MOC exam.

Original languageEnglish (US)
Pages (from-to)1369-1375
Number of pages7
JournalAcademic Medicine
Volume94
Issue number9
DOIs
StatePublished - Sep 1 2019

ASJC Scopus subject areas

  • Education

Fingerprint

Dive into the research topics of 'Incorporating Physician Input into a Maintenance of Certification Examination: A Content Validity Tool'. Together they form a unique fingerprint.

Cite this