Case Matching and the Reduction of Selection Bias in Quasi-Experiments: The Relative Importance of Pretest Measures of Outcome, of Unreliable Measurement, and of Mode of Data Analysis

Thomas D. Cook*, Peter M. Steiner

*Corresponding author for this work

Research output: Contribution to journalArticle

51 Scopus citations

Abstract

In this article, we note the many ontological, epistemological, and methodological similarities between how Campbell and Rubin conceptualize causation. We then explore 3 differences in their written emphases about individual case matching in observational studies. We contend that (a) Campbell places greater emphasis than Rubin on the special role of pretest measures of outcome among matching variables; (b) Campbell is more explicitly concerned with unreliability in the covariates; and (c) for analyzing the outcome, only Rubin emphasizes the advantages of using propensity score over regression methods. To explore how well these 3 factors reduce bias, we reanalyze and review within-study comparisons that contrast experimental and statistically adjusted nonexperimental causal estimates from studies with the same target population and treatment content. In this context, the choice of covariates counts most for reducing selection bias, and the pretest usually plays a special role relative to all the other covariates considered singly. Unreliability in the covariates also influences bias reduction but by less. Furthermore, propensity score and regression methods produce comparable degrees of bias reduction, though these within-study comparisons may not have met the theoretically specified conditions most likely to produce differences due to analytic method.

Original languageEnglish (US)
Pages (from-to)56-68
Number of pages13
JournalPsychological Methods
Volume15
Issue number1
DOIs
StatePublished - Mar 1 2010

Keywords

  • bias reduction
  • multiple regression
  • pretests
  • unreliability propensity scores

ASJC Scopus subject areas

  • Psychology (miscellaneous)

Fingerprint Dive into the research topics of 'Case Matching and the Reduction of Selection Bias in Quasi-Experiments: The Relative Importance of Pretest Measures of Outcome, of Unreliable Measurement, and of Mode of Data Analysis'. Together they form a unique fingerprint.

  • Cite this