On the Convexity of ROC Curves Estimated from Radiological Test Results

Lorenzo L. Pesce, Charles E. Metz*, Kevin S. Berbaum

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

24 Scopus citations

Abstract

Rationale and Objectives: Although an ideal observer's receiver operating characteristic (ROC) curve must be convex-ie, its slope must decrease monotonically-published fits to empirical data often display " hooks." Such fits sometimes are accepted on the basis of an argument that experiments are done with real, rather than ideal, observers. However, the fact that ideal observers must produce convex curves does not imply that convex curves describe only ideal observers. This article aims to identify the practical implications of nonconvex ROC curves and the conditions that can lead to empirical or fitted ROC curves that are not convex. Materials and Methods: This article views nonconvex ROC curves from historical, theoretical, and statistical perspectives, which we describe briefly. We then consider population ROC curves with various shapes and analyze the types of medical decisions that they imply. Finally, we describe how sampling variability and curve-fitting algorithms can produce ROC curve estimates that include hooks. Results: We show that hooks in population ROC curves imply the use of an irrational decision strategy, even when the curve does not cross the chance line, and therefore usually are untenable in medical settings. Moreover, we sketch a simple approach to improve any nonconvex ROC curve by adding statistical variation to the decision process. Finally, we sketch how to test whether hooks present in ROC data are likely to have been caused by chance alone and how some hooked ROCs found in the literature can be easily explained as fitting artifacts or modeling issues. Conclusion: In general, ROC curve fits that show hooks should be looked on with suspicion unless other arguments justify their presence.

Original languageEnglish (US)
Pages (from-to)960-968.e4
JournalAcademic radiology
Volume17
Issue number8
DOIs
StatePublished - Aug 2010

Keywords

  • Contaminated ROC model
  • Maximum likelihood estimation (MLE)
  • Proper ROC curve
  • Receiver operating characteristic (ROC) analysis

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Fingerprint Dive into the research topics of 'On the Convexity of ROC Curves Estimated from Radiological Test Results'. Together they form a unique fingerprint.

Cite this