Using latent variable modeling and multiple imputation to calibrate rater bias in diagnosis assessment

Juned Siddique*, Catherine M. Crespi, Robert D. Gibbons, Bonnie L. Green

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


We present an approach that uses latent variable modeling and multiple imputation to correct rater bias when one group of raters tends to be more lenient in assigning a diagnosis than another. Our method assumes that there exists an unobserved moderate category of patient who is assigned a positive diagnosis by one type of rater and a negative diagnosis by the other type. We present a Bayesian random effects censored ordinal probit model that allows us to calibrate the diagnoses across rater types by identifying and multiply imputing 'case' or 'non-case' status for patients in the moderate category. A Markov chain Monte Carlo algorithm is presented to estimate the posterior distribution of the model parameters and generate multiple imputations. Our method enables the calibrated diagnosis variable to be used in subsequent analyses while also preserving uncertainty in true diagnosis. We apply our model to diagnoses of posttraumatic stress disorder (PTSD) from a depression study where nurse practitioners were twice as likely as clinical psychologists to diagnose PTSD despite the fact that participants were randomly assigned to either a nurse or a psychologist. Our model appears to balance PTSD rates across raters, provides a good fit to the data, and preserves between-rater variability. After calibrating the diagnoses of PTSD across rater types, we perform an analysis looking at the effects of comorbid PTSD on changes in depression scores over time. Results are compared with an analysis that uses the original diagnoses and show that calibrating the PTSD diagnoses can yield different inferences.

Original languageEnglish (US)
Pages (from-to)160-174
Number of pages15
JournalStatistics in Medicine
Issue number2
StatePublished - Jan 30 2011


  • Censoring
  • Leniency error
  • Markov chain Monte Carlo
  • Ordinal probit
  • PTSD

ASJC Scopus subject areas

  • Epidemiology
  • Statistics and Probability


Dive into the research topics of 'Using latent variable modeling and multiple imputation to calibrate rater bias in diagnosis assessment'. Together they form a unique fingerprint.

Cite this