On the importance of reliable covariate measurement in selection bias adjustments using propensity scores

Peter M. Steiner, Thomas D. Cook, William R. Shadish

Research output: Contribution to journalArticlepeer-review

74 Scopus citations

Abstract

The effect of unreliability of measurement on propensity score (PS) adjusted treatment effects has not been previously studied. The authors report on a study simulating different degrees of unreliability in the multiple covariates that were used to estimate the PS. The simulation uses the same data as two prior studies. Shadish, Clark, and Steiner showed that a PS formed from many covariates demonstrably reduced selection bias, while Steiner, Cook, Shadish, and Clark identified the subsets of covariates from the larger set that were most effective for bias reduction. Adding different degrees of random error to these covariates in a simulation, the authors demonstrate that unreliability of measurement can degrade the ability of PSs to reduce bias. Specifically, increases in reliability only promote bias reduction, if the covariates are effective in reducing bias to begin with. Increasing or decreasing the reliability of covariates that do not effectively reduce selection bias makes no difference at all.

Original languageEnglish (US)
Pages (from-to)213-236
Number of pages24
JournalJournal of Educational and Behavioral Statistics
Volume36
Issue number2
DOIs
StatePublished - Jun 2011

Keywords

  • attenuation bias
  • measurement error
  • propensity score
  • selection bias
  • strong ignorability

ASJC Scopus subject areas

  • Education
  • Social Sciences (miscellaneous)

Fingerprint Dive into the research topics of 'On the importance of reliable covariate measurement in selection bias adjustments using propensity scores'. Together they form a unique fingerprint.

Cite this