Estimating screening-mammography receiver operating characteristic (ROC) curves from stratified random samples of screening mammograms

A simulation study.

Richard M. Zur*, Lorenzo Luigi Pesce, Yulei Jiang

*Corresponding author for this work

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Rationale and Objectives: To evaluate stratified random sampling (SRS) of screening mammograms by (1) Breast Imaging Reporting and Data System (BI-RADS) assessment categories, and (2) the presence of breast cancer in mammograms, for estimation of screening-mammography receiver operating characteristic (ROC) curves in retrospective observer studies. Materials and Methods: We compared observer study case sets constructed by (1) random sampling (RS); (2) SRS with proportional allocation (SRS-P) with BI-RADS 1 and 2 noncancer cases accounting for 90.6% of all noncancer cases; (3) SRS with disproportional allocation (SRS-D) with BI-RADS 1 and 2 noncancer cases accounting for 10%-80%; and (4) SRS-D and multiple imputation (SRS-D+MI) with missing BI-RADS 1 and 2 noncancer cases imputed to recover the 90.6% proportion. Monte Carlo simulated case sets were drawn from a large case population modeled after published Digital Mammography Imaging Screening Trial data. We compared the bias, root-mean-square error, and coverage of 95% confidence intervals of area under the ROC curve (AUC) estimates from the sampling methods (200-2000 cases, of which 25% were cancer cases) versus from the large case population. Results: AUC estimates were unbiased from RS, SRS-P, and SRS-D+MI, but biased from SRS-D. AUC estimates from SRS-P and SRS-D+MI had 10% smaller root-mean-square error than RS. Conclusions: Both SRS-P and SRS-D+MI can be used to obtain unbiased and 10% more efficient estimate of screening-mammography ROC curves.

Original languageEnglish (US)
Pages (from-to)580-590
Number of pages11
JournalAcademic Radiology
Volume22
Issue number5
DOIs
StatePublished - May 1 2015

Fingerprint

Mammography
ROC Curve
Information Systems
Breast
Random Allocation
Area Under Curve
Selection Bias
Population
Retrospective Studies
Confidence Intervals
Breast Neoplasms
Neoplasms

Keywords

  • Observer studies
  • ROC analysis
  • Screening mammography
  • Simulation study
  • Stratified random sampling

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this

@article{bacacb6be616405d929246423dac7980,
title = "Estimating screening-mammography receiver operating characteristic (ROC) curves from stratified random samples of screening mammograms: A simulation study.",
abstract = "Rationale and Objectives: To evaluate stratified random sampling (SRS) of screening mammograms by (1) Breast Imaging Reporting and Data System (BI-RADS) assessment categories, and (2) the presence of breast cancer in mammograms, for estimation of screening-mammography receiver operating characteristic (ROC) curves in retrospective observer studies. Materials and Methods: We compared observer study case sets constructed by (1) random sampling (RS); (2) SRS with proportional allocation (SRS-P) with BI-RADS 1 and 2 noncancer cases accounting for 90.6{\%} of all noncancer cases; (3) SRS with disproportional allocation (SRS-D) with BI-RADS 1 and 2 noncancer cases accounting for 10{\%}-80{\%}; and (4) SRS-D and multiple imputation (SRS-D+MI) with missing BI-RADS 1 and 2 noncancer cases imputed to recover the 90.6{\%} proportion. Monte Carlo simulated case sets were drawn from a large case population modeled after published Digital Mammography Imaging Screening Trial data. We compared the bias, root-mean-square error, and coverage of 95{\%} confidence intervals of area under the ROC curve (AUC) estimates from the sampling methods (200-2000 cases, of which 25{\%} were cancer cases) versus from the large case population. Results: AUC estimates were unbiased from RS, SRS-P, and SRS-D+MI, but biased from SRS-D. AUC estimates from SRS-P and SRS-D+MI had 10{\%} smaller root-mean-square error than RS. Conclusions: Both SRS-P and SRS-D+MI can be used to obtain unbiased and 10{\%} more efficient estimate of screening-mammography ROC curves.",
keywords = "Observer studies, ROC analysis, Screening mammography, Simulation study, Stratified random sampling",
author = "Zur, {Richard M.} and Pesce, {Lorenzo Luigi} and Yulei Jiang",
year = "2015",
month = "5",
day = "1",
doi = "10.1016/j.acra.2014.12.011",
language = "English (US)",
volume = "22",
pages = "580--590",
journal = "Academic Radiology",
issn = "1076-6332",
publisher = "Elsevier USA",
number = "5",

}

Estimating screening-mammography receiver operating characteristic (ROC) curves from stratified random samples of screening mammograms : A simulation study. / Zur, Richard M.; Pesce, Lorenzo Luigi; Jiang, Yulei.

In: Academic Radiology, Vol. 22, No. 5, 01.05.2015, p. 580-590.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Estimating screening-mammography receiver operating characteristic (ROC) curves from stratified random samples of screening mammograms

T2 - A simulation study.

AU - Zur, Richard M.

AU - Pesce, Lorenzo Luigi

AU - Jiang, Yulei

PY - 2015/5/1

Y1 - 2015/5/1

N2 - Rationale and Objectives: To evaluate stratified random sampling (SRS) of screening mammograms by (1) Breast Imaging Reporting and Data System (BI-RADS) assessment categories, and (2) the presence of breast cancer in mammograms, for estimation of screening-mammography receiver operating characteristic (ROC) curves in retrospective observer studies. Materials and Methods: We compared observer study case sets constructed by (1) random sampling (RS); (2) SRS with proportional allocation (SRS-P) with BI-RADS 1 and 2 noncancer cases accounting for 90.6% of all noncancer cases; (3) SRS with disproportional allocation (SRS-D) with BI-RADS 1 and 2 noncancer cases accounting for 10%-80%; and (4) SRS-D and multiple imputation (SRS-D+MI) with missing BI-RADS 1 and 2 noncancer cases imputed to recover the 90.6% proportion. Monte Carlo simulated case sets were drawn from a large case population modeled after published Digital Mammography Imaging Screening Trial data. We compared the bias, root-mean-square error, and coverage of 95% confidence intervals of area under the ROC curve (AUC) estimates from the sampling methods (200-2000 cases, of which 25% were cancer cases) versus from the large case population. Results: AUC estimates were unbiased from RS, SRS-P, and SRS-D+MI, but biased from SRS-D. AUC estimates from SRS-P and SRS-D+MI had 10% smaller root-mean-square error than RS. Conclusions: Both SRS-P and SRS-D+MI can be used to obtain unbiased and 10% more efficient estimate of screening-mammography ROC curves.

AB - Rationale and Objectives: To evaluate stratified random sampling (SRS) of screening mammograms by (1) Breast Imaging Reporting and Data System (BI-RADS) assessment categories, and (2) the presence of breast cancer in mammograms, for estimation of screening-mammography receiver operating characteristic (ROC) curves in retrospective observer studies. Materials and Methods: We compared observer study case sets constructed by (1) random sampling (RS); (2) SRS with proportional allocation (SRS-P) with BI-RADS 1 and 2 noncancer cases accounting for 90.6% of all noncancer cases; (3) SRS with disproportional allocation (SRS-D) with BI-RADS 1 and 2 noncancer cases accounting for 10%-80%; and (4) SRS-D and multiple imputation (SRS-D+MI) with missing BI-RADS 1 and 2 noncancer cases imputed to recover the 90.6% proportion. Monte Carlo simulated case sets were drawn from a large case population modeled after published Digital Mammography Imaging Screening Trial data. We compared the bias, root-mean-square error, and coverage of 95% confidence intervals of area under the ROC curve (AUC) estimates from the sampling methods (200-2000 cases, of which 25% were cancer cases) versus from the large case population. Results: AUC estimates were unbiased from RS, SRS-P, and SRS-D+MI, but biased from SRS-D. AUC estimates from SRS-P and SRS-D+MI had 10% smaller root-mean-square error than RS. Conclusions: Both SRS-P and SRS-D+MI can be used to obtain unbiased and 10% more efficient estimate of screening-mammography ROC curves.

KW - Observer studies

KW - ROC analysis

KW - Screening mammography

KW - Simulation study

KW - Stratified random sampling

UR - http://www.scopus.com/inward/record.url?scp=84927056845&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84927056845&partnerID=8YFLogxK

U2 - 10.1016/j.acra.2014.12.011

DO - 10.1016/j.acra.2014.12.011

M3 - Article

VL - 22

SP - 580

EP - 590

JO - Academic Radiology

JF - Academic Radiology

SN - 1076-6332

IS - 5

ER -