Stratified sampling for case selection criteria for evaluating CAD

Robert M. Nishikawa, Lorenzo L. Pesce

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Ideally, the outcome of any CAD performance assessment should predict how well the system would work if used clinically. In principle, if the selection process draws cases that are "representative" of the general patient population, the study design will be unbiased. In this study we explored the effect of stratified sampling on stand-alone and radiologists' performance using data from an observer study. Although our database was relatively small, 50 cancer cases, no meaningful difference in performance was measured among different stratified sampling schemes or against the whole dataset nor was there any difference in the variance in the measured performance metrics. These results cast doubts on the usefulness of requiring stratified sampling, whose added cost does not seem to be justifiable without empirical evidence. We believe that it is more important to specify how cases should be collected than try to define the range and frequency of the characteristics of patients and cancers to be included the dataset, which we suspect to be prone to actually produce unrealistic samples.

Original languageEnglish (US)
Title of host publicationDigital Mammography - 10th International Workshop, IWDM 2010, Proceedings
Number of pages6
StatePublished - 2010
Event10th International Workshop on Digital Mammography, IWDM 2010 - Girona, Catalonia, Spain
Duration: Jun 16 2010Jun 18 2010

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume6136 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference10th International Workshop on Digital Mammography, IWDM 2010
CityGirona, Catalonia


  • breast cancer
  • case selection
  • computer-aided detection
  • computer-aided diagnosis
  • evaluation
  • mammography
  • observer study

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Stratified sampling for case selection criteria for evaluating CAD'. Together they form a unique fingerprint.

Cite this