Stratified Sampling Using Cluster Analysis: A Sample Selection Strategy for Improved Generalizations From Experiments

Elizabeth Tipton*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

26 Scopus citations

Abstract

Background: An important question in the design of experiments is howto ensure that the findings from the experiment are generalizable to a largerpopulation. This concern with generalizability is particularly importantwhen treatment effects are heterogeneous and when selecting units intothe experiment using random sampling is not possible-two conditionscommonly met in large-scale educational experiments. Method: Thisarticle introduces a model-based balanced-sampling framework for improvinggeneralizations, with a focus on developing methods that are robust tomodel misspecification. Additionally, the article provides a new method forsample selection within this framework: First units in an inference populationare divided into relatively homogenous strata using cluster analysis, andthen the sample is selected using distance rankings. Result: In order todemonstrate and evaluate the method, a reanalysis of a completedexperiment is conducted. This example compares samples selected usingthe new method with the actual sample used in the experiment. Resultsindicate that even under high nonresponse, balance is better on mostcovariates and that fewer coverage errors result. Conclusion: The articleconcludes with a discussion of additional benefits and limitations of themethod.

Original languageEnglish (US)
Pages (from-to)109-139
Number of pages31
JournalEvaluation Review
Volume37
Issue number2
DOIs
StatePublished - Apr 2013

Keywords

  • cluster analysis
  • experimental design
  • external validity
  • model-based sampling
  • stratified sampling
  • treatment effect heterogeneity

ASJC Scopus subject areas

  • Arts and Humanities (miscellaneous)
  • Social Sciences(all)

Fingerprint Dive into the research topics of 'Stratified Sampling Using Cluster Analysis: A Sample Selection Strategy for Improved Generalizations From Experiments'. Together they form a unique fingerprint.

Cite this