The importance of covariate selection in controlling for selection bias in observational studies

Peter M. Steiner*, Thomas D. Cook, William R. Shadish, M. H. Clark

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

328 Scopus citations

Abstract

The assumption of strongly ignorable treatment assignment is required for eliminating selection bias in observational studies. To meet this assumption, researchers often rely on a strategy of selecting covariates that they think will control for selection bias. Theory indicates that the most important covariates are those highly correlated with both the real selection process and the potential outcomes. However, when planning a study, it is rarely possible to identify such covariates with certainty. In this article, we report on an extensive reanalysis of a within-study comparison that contrasts a randomized experiment and a quasi-experiment. Various covariate sets were used to adjust for initial group differences in the quasi-experiment that was characterized by self-selection into treatment. The adjusted effect sizes were then compared with the experimental ones to identify which individual covariates, and which conceptually grouped sets of covariates, were responsible for the high degree of bias reduction achieved in the adjusted quasi-experiment. Such results provide strong clues about preferred strategies for identifying the covariates most likely to reduce bias when planning a study and when the true selection process is not known.

Original languageEnglish (US)
Pages (from-to)250-267
Number of pages18
JournalPsychological methods
Volume15
Issue number3
DOIs
StatePublished - Sep 2010

Keywords

  • Hidden bias
  • Propensity score
  • Selection bias
  • Strong ignorability
  • Within-study comparison

ASJC Scopus subject areas

  • Psychology (miscellaneous)

Fingerprint

Dive into the research topics of 'The importance of covariate selection in controlling for selection bias in observational studies'. Together they form a unique fingerprint.

Cite this