Unbiased causal inference from an observational study: Results of a within-study comparison

Steffi Pohl*, Peter M. Steiner, Jens Eisermann, Renate Soellner, Thomas D. Cook

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

32 Scopus citations

Abstract

Adjustment methods such as propensity scores and analysis of covariance are often used for estimating treatment effects in nonexperimental data. Shadish, Clark, and Steiner used a within-study comparison to test how well these adjustments work in practice. They randomly assigned participating students to a randomized or nonrandomized experiment. Treatment effects were then estimated in the experiment and compared to the adjusted nonexperimental estimates. Most of the selection bias in the nonexperiment was reduced. The present study replicates the study of Shadishet al.despite some differences in design and in the size and direction of the initial bias. The results show that the selection of covariates matters considerably for bias reduction in nonexperiments but that the choice of analysis matters less.

Original languageEnglish (US)
Pages (from-to)463-479
Number of pages17
JournalEducational Evaluation and Policy Analysis
Volume31
Issue number4
DOIs
StatePublished - Dec 2009

Keywords

  • Analysis of covariance
  • Causal inference
  • Observational study
  • Propensity scores
  • Within-study comparison

ASJC Scopus subject areas

  • Education

Fingerprint Dive into the research topics of 'Unbiased causal inference from an observational study: Results of a within-study comparison'. Together they form a unique fingerprint.

Cite this