A Bayesian interpretation of the "proper" binormal ROC model using a uniform prior distribution for the area under the curve

Richard M. Zur*, Lorenzo Luigi Pesce, Yulei Jiang, Charles E. Metz

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Maximum likelihood estimation of receiver operating characteristic (ROC) curves using the "proper" binormal model can be interpreted in terms of Bayesian estimation as assuming a flat joint prior distribution on the c and d a parameters However, this is equivalent to assuming a non-flat prior distribution for the area under the curve (AUC) that peaks at AUC - 1.0. We hypothesize that this implicit prior on AUC biases the maximum likelihood estimate (MLE) of AUC We propose a Bayesian implementation of the "proper" binomial ROC curve-fitting model with a prior distribution that is marginally flat on AUC and conditionally flat over c. This specifies a non-flat joint prior for c and d a We developed a Monte Carlo Markov chain (MCMC) algorithm to estimate the posterior distribution and the maximum a posteriori (MAP) estimate of AUC. We performed a simulation study using 500 draws of a small dataset (25 normal and 25 abnormal cases) with an underlying AUC value of 0.85. When the prior distribution was a flat joint prior on c and d a the MLE and MAP estimates agreed, suggesting that the MCMC algorithm worked correctly. When the prior distribution was marginally flat on AUC, the MAP estimate of AUC appeared to be biased low. However the MAP estimate of AUC for perfectly separable degenerate datasets did not appear to be biased. Further work is needed to validate the algorithm and refine the prior assumptions.

Original languageEnglish (US)
Title of host publicationMedical Imaging 2007
Subtitle of host publicationImage Perception, Observer Performance, and Technology Assessment
DOIs
StatePublished - Oct 15 2007
EventMedical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment - San Diego, CA, United States
Duration: Feb 21 2007Feb 22 2007

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
Volume6515
ISSN (Print)1605-7422

Conference

ConferenceMedical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment
CountryUnited States
CitySan Diego, CA
Period2/21/072/22/07

Fingerprint

ROC Curve
Area Under Curve
receivers
Markov processes
Maximum likelihood
curves
Maximum likelihood estimation
Curve fitting
Likelihood Functions
maximum likelihood estimates
Markov Chains
estimates
Markov chains
Joints
curve fitting

Keywords

  • "Proper" receiver operating characteristic curve analysis
  • Bayesian estimation
  • Monte Carlo Markov chains

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Atomic and Molecular Physics, and Optics
  • Biomaterials
  • Radiology Nuclear Medicine and imaging

Cite this

Zur, R. M., Pesce, L. L., Jiang, Y., & Metz, C. E. (2007). A Bayesian interpretation of the "proper" binormal ROC model using a uniform prior distribution for the area under the curve. In Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment [651511] (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 6515). https://doi.org/10.1117/12.711689
Zur, Richard M. ; Pesce, Lorenzo Luigi ; Jiang, Yulei ; Metz, Charles E. / A Bayesian interpretation of the "proper" binormal ROC model using a uniform prior distribution for the area under the curve. Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment. 2007. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE).
@inproceedings{a5c989e0a0df499a9c3c731213136052,
title = "A Bayesian interpretation of the {"}proper{"} binormal ROC model using a uniform prior distribution for the area under the curve",
abstract = "Maximum likelihood estimation of receiver operating characteristic (ROC) curves using the {"}proper{"} binormal model can be interpreted in terms of Bayesian estimation as assuming a flat joint prior distribution on the c and d a parameters However, this is equivalent to assuming a non-flat prior distribution for the area under the curve (AUC) that peaks at AUC - 1.0. We hypothesize that this implicit prior on AUC biases the maximum likelihood estimate (MLE) of AUC We propose a Bayesian implementation of the {"}proper{"} binomial ROC curve-fitting model with a prior distribution that is marginally flat on AUC and conditionally flat over c. This specifies a non-flat joint prior for c and d a We developed a Monte Carlo Markov chain (MCMC) algorithm to estimate the posterior distribution and the maximum a posteriori (MAP) estimate of AUC. We performed a simulation study using 500 draws of a small dataset (25 normal and 25 abnormal cases) with an underlying AUC value of 0.85. When the prior distribution was a flat joint prior on c and d a the MLE and MAP estimates agreed, suggesting that the MCMC algorithm worked correctly. When the prior distribution was marginally flat on AUC, the MAP estimate of AUC appeared to be biased low. However the MAP estimate of AUC for perfectly separable degenerate datasets did not appear to be biased. Further work is needed to validate the algorithm and refine the prior assumptions.",
keywords = "{"}Proper{"} receiver operating characteristic curve analysis, Bayesian estimation, Monte Carlo Markov chains",
author = "Zur, {Richard M.} and Pesce, {Lorenzo Luigi} and Yulei Jiang and Metz, {Charles E.}",
year = "2007",
month = "10",
day = "15",
doi = "10.1117/12.711689",
language = "English (US)",
isbn = "0819466336",
series = "Progress in Biomedical Optics and Imaging - Proceedings of SPIE",
booktitle = "Medical Imaging 2007",

}

Zur, RM, Pesce, LL, Jiang, Y & Metz, CE 2007, A Bayesian interpretation of the "proper" binormal ROC model using a uniform prior distribution for the area under the curve. in Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment., 651511, Progress in Biomedical Optics and Imaging - Proceedings of SPIE, vol. 6515, Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment, San Diego, CA, United States, 2/21/07. https://doi.org/10.1117/12.711689

A Bayesian interpretation of the "proper" binormal ROC model using a uniform prior distribution for the area under the curve. / Zur, Richard M.; Pesce, Lorenzo Luigi; Jiang, Yulei; Metz, Charles E.

Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment. 2007. 651511 (Progress in Biomedical Optics and Imaging - Proceedings of SPIE; Vol. 6515).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A Bayesian interpretation of the "proper" binormal ROC model using a uniform prior distribution for the area under the curve

AU - Zur, Richard M.

AU - Pesce, Lorenzo Luigi

AU - Jiang, Yulei

AU - Metz, Charles E.

PY - 2007/10/15

Y1 - 2007/10/15

N2 - Maximum likelihood estimation of receiver operating characteristic (ROC) curves using the "proper" binormal model can be interpreted in terms of Bayesian estimation as assuming a flat joint prior distribution on the c and d a parameters However, this is equivalent to assuming a non-flat prior distribution for the area under the curve (AUC) that peaks at AUC - 1.0. We hypothesize that this implicit prior on AUC biases the maximum likelihood estimate (MLE) of AUC We propose a Bayesian implementation of the "proper" binomial ROC curve-fitting model with a prior distribution that is marginally flat on AUC and conditionally flat over c. This specifies a non-flat joint prior for c and d a We developed a Monte Carlo Markov chain (MCMC) algorithm to estimate the posterior distribution and the maximum a posteriori (MAP) estimate of AUC. We performed a simulation study using 500 draws of a small dataset (25 normal and 25 abnormal cases) with an underlying AUC value of 0.85. When the prior distribution was a flat joint prior on c and d a the MLE and MAP estimates agreed, suggesting that the MCMC algorithm worked correctly. When the prior distribution was marginally flat on AUC, the MAP estimate of AUC appeared to be biased low. However the MAP estimate of AUC for perfectly separable degenerate datasets did not appear to be biased. Further work is needed to validate the algorithm and refine the prior assumptions.

AB - Maximum likelihood estimation of receiver operating characteristic (ROC) curves using the "proper" binormal model can be interpreted in terms of Bayesian estimation as assuming a flat joint prior distribution on the c and d a parameters However, this is equivalent to assuming a non-flat prior distribution for the area under the curve (AUC) that peaks at AUC - 1.0. We hypothesize that this implicit prior on AUC biases the maximum likelihood estimate (MLE) of AUC We propose a Bayesian implementation of the "proper" binomial ROC curve-fitting model with a prior distribution that is marginally flat on AUC and conditionally flat over c. This specifies a non-flat joint prior for c and d a We developed a Monte Carlo Markov chain (MCMC) algorithm to estimate the posterior distribution and the maximum a posteriori (MAP) estimate of AUC. We performed a simulation study using 500 draws of a small dataset (25 normal and 25 abnormal cases) with an underlying AUC value of 0.85. When the prior distribution was a flat joint prior on c and d a the MLE and MAP estimates agreed, suggesting that the MCMC algorithm worked correctly. When the prior distribution was marginally flat on AUC, the MAP estimate of AUC appeared to be biased low. However the MAP estimate of AUC for perfectly separable degenerate datasets did not appear to be biased. Further work is needed to validate the algorithm and refine the prior assumptions.

KW - "Proper" receiver operating characteristic curve analysis

KW - Bayesian estimation

KW - Monte Carlo Markov chains

UR - http://www.scopus.com/inward/record.url?scp=35148845650&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=35148845650&partnerID=8YFLogxK

U2 - 10.1117/12.711689

DO - 10.1117/12.711689

M3 - Conference contribution

AN - SCOPUS:35148845650

SN - 0819466336

SN - 9780819466334

T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE

BT - Medical Imaging 2007

ER -

Zur RM, Pesce LL, Jiang Y, Metz CE. A Bayesian interpretation of the "proper" binormal ROC model using a uniform prior distribution for the area under the curve. In Medical Imaging 2007: Image Perception, Observer Performance, and Technology Assessment. 2007. 651511. (Progress in Biomedical Optics and Imaging - Proceedings of SPIE). https://doi.org/10.1117/12.711689