Estimating screening-mammography receiver operating characteristic (ROC) curves from stratified random samples of screening mammograms: A simulation study.

Richard M. Zur*, Lorenzo Luigi Pesce, Yulei Jiang

*Corresponding author for this work

Research output: Contribution to journalArticle

1 Scopus citations


Rationale and Objectives: To evaluate stratified random sampling (SRS) of screening mammograms by (1) Breast Imaging Reporting and Data System (BI-RADS) assessment categories, and (2) the presence of breast cancer in mammograms, for estimation of screening-mammography receiver operating characteristic (ROC) curves in retrospective observer studies. Materials and Methods: We compared observer study case sets constructed by (1) random sampling (RS); (2) SRS with proportional allocation (SRS-P) with BI-RADS 1 and 2 noncancer cases accounting for 90.6% of all noncancer cases; (3) SRS with disproportional allocation (SRS-D) with BI-RADS 1 and 2 noncancer cases accounting for 10%-80%; and (4) SRS-D and multiple imputation (SRS-D+MI) with missing BI-RADS 1 and 2 noncancer cases imputed to recover the 90.6% proportion. Monte Carlo simulated case sets were drawn from a large case population modeled after published Digital Mammography Imaging Screening Trial data. We compared the bias, root-mean-square error, and coverage of 95% confidence intervals of area under the ROC curve (AUC) estimates from the sampling methods (200-2000 cases, of which 25% were cancer cases) versus from the large case population. Results: AUC estimates were unbiased from RS, SRS-P, and SRS-D+MI, but biased from SRS-D. AUC estimates from SRS-P and SRS-D+MI had 10% smaller root-mean-square error than RS. Conclusions: Both SRS-P and SRS-D+MI can be used to obtain unbiased and 10% more efficient estimate of screening-mammography ROC curves.

Original languageEnglish (US)
Pages (from-to)580-590
Number of pages11
JournalAcademic Radiology
Issue number5
StatePublished - May 1 2015



  • Observer studies
  • ROC analysis
  • Screening mammography
  • Simulation study
  • Stratified random sampling

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging

Cite this