Generalizing from unrepresentative experiments: A stratified propensity score approach

Colm O'Muircheartaigh*, Larry V. Hedges

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

84 Scopus citations

Abstract

The paper addresses means of generalizing from an experiment based on a non-probability sample to a population of interest and to subpopulations of interest, where information is available about relevant covariates in the whole population. Using stratification based on propensity score matching with an external populationwide data set, an estimator of the population average treatment effect is constructed. An example is presented in which the applicability of a major education intervention in a non-probability sample of schools in Texas, USA, is assessed for the state as a whole and for its constituent counties. The implications of the results are discussed for two important situations: how to use this methodology to establish where future experiments should be conducted to improve this generalization and how to construct a priori a strategy for experimentation which will maximize both the initial inferential power and the final inferential basis for a series of experiments.

Original languageEnglish (US)
Pages (from-to)195-210
Number of pages16
JournalJournal of the Royal Statistical Society. Series C: Applied Statistics
Volume63
Issue number2
DOIs
StatePublished - Feb 2014

Keywords

  • Generalization
  • Non-probability samples
  • Propensity score stratification

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Fingerprint

Dive into the research topics of 'Generalizing from unrepresentative experiments: A stratified propensity score approach'. Together they form a unique fingerprint.

Cite this