Interpreting incremental value of markers added to risk prediction models.

Michael J. Pencina*, Ralph B. D'Agostino, Karol M. Pencina, A. Cecile J W Janssens, Philip Greenland

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

258 Scopus citations

Abstract

The discrimination of a risk prediction model measures that model's ability to distinguish between subjects with and without events. The area under the receiver operating characteristic curve (AUC) is a popular measure of discrimination. However, the AUC has recently been criticized for its insensitivity in model comparisons in which the baseline model has performed well. Thus, 2 other measures have been proposed to capture improvement in discrimination for nested models: the integrated discrimination improvement and the continuous net reclassification improvement. In the present study, the authors use mathematical relations and numerical simulations to quantify the improvement in discrimination offered by candidate markers of different strengths as measured by their effect sizes. They demonstrate that the increase in the AUC depends on the strength of the baseline model, which is true to a lesser degree for the integrated discrimination improvement. On the other hand, the continuous net reclassification improvement depends only on the effect size of the candidate variable and its correlation with other predictors. These measures are illustrated using the Framingham model for incident atrial fibrillation. The authors conclude that the increase in the AUC, integrated discrimination improvement, and net reclassification improvement offer complementary information and thus recommend reporting all 3 alongside measures characterizing the performance of the final model.

Original languageEnglish (US)
Pages (from-to)473-481
Number of pages9
JournalAmerican journal of epidemiology
Volume176
Issue number6
DOIs
StatePublished - Sep 15 2012

ASJC Scopus subject areas

  • Epidemiology

Fingerprint Dive into the research topics of 'Interpreting incremental value of markers added to risk prediction models.'. Together they form a unique fingerprint.

Cite this