Generalized linear mixed models for multi-reader multi-case studies of diagnostic tests

Wei Liu, Norberto Pantoja-Galicia, Bo Zhang, Richard M. Kotz, Gene Pennello, Hui Zhang, Jessie Jacob, Zhiwei Zhang*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

4 Scopus citations


Diagnostic tests are often compared in multi-reader multi-case (MRMC) studies in which a number of cases (subjects with or without the disease in question) are examined by several readers using all tests to be compared. One of the commonly used methods for analyzing MRMC data is the Obuchowski-Rockette (OR) method, which assumes that the true area under the receiver operating characteristic curve (AUC) for each combination of reader and test follows a linear mixed model with fixed effects for test and random effects for reader and the reader-test interaction. This article proposes generalized linear mixed models which generalize the OR model by incorporating a range-appropriate link function that constrains the true AUCs to the unit interval. The proposed models can be estimated by maximizing a pseudo-likelihood based on the approximate normality of AUC estimates. A Monte Carlo expectation-maximization algorithm can be used to maximize the pseudo-likelihood, and a non-parametric bootstrap procedure can be used for inference. The proposed method is evaluated in a simulation study and applied to an MRMC study of breast cancer detection.

Original languageEnglish (US)
Pages (from-to)1373-1388
Number of pages16
JournalStatistical Methods in Medical Research
Issue number3
StatePublished - Jun 1 2017


  • AUC
  • EM algorithm
  • ROC curve
  • diagnostic medicine
  • pseudo-likelihood
  • random effect

ASJC Scopus subject areas

  • Epidemiology
  • Statistics and Probability
  • Health Information Management


Dive into the research topics of 'Generalized linear mixed models for multi-reader multi-case studies of diagnostic tests'. Together they form a unique fingerprint.

Cite this