Generalizing from unrepresentative experiments: A stratified propensity score approach

Colm O'Muircheartaigh*, Larry V. Hedges

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

67 Scopus citations


The paper addresses means of generalizing from an experiment based on a non-probability sample to a population of interest and to subpopulations of interest, where information is available about relevant covariates in the whole population. Using stratification based on propensity score matching with an external populationwide data set, an estimator of the population average treatment effect is constructed. An example is presented in which the applicability of a major education intervention in a non-probability sample of schools in Texas, USA, is assessed for the state as a whole and for its constituent counties. The implications of the results are discussed for two important situations: how to use this methodology to establish where future experiments should be conducted to improve this generalization and how to construct a priori a strategy for experimentation which will maximize both the initial inferential power and the final inferential basis for a series of experiments.

Original languageEnglish (US)
Pages (from-to)195-210
Number of pages16
JournalJournal of the Royal Statistical Society. Series C: Applied Statistics
Issue number2
StatePublished - Feb 2014


  • Generalization
  • Non-probability samples
  • Propensity score stratification

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty


Dive into the research topics of 'Generalizing from unrepresentative experiments: A stratified propensity score approach'. Together they form a unique fingerprint.

Cite this