Site Selection in Experiments: An Assessment of Site Recruitment and Generalizability in Two Scale-up Studies

Elizabeth Tipton*, Lauren Fellers, Sarah Caverly, Michael Vaden-Kiernan, Geoffrey Borman, Kate Sullivan, Veronica Ruiz de Castilla

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

Recently, statisticians have begun developing methods to improve the generalizability of results from large-scale experiments in education. This work has included the development of methods for improved site selection when random sampling is infeasible, including the use of stratification and targeted recruitment strategies. This article provides the next step in this literature—a template for assessing generalizability after a study is completed. In this template, first records from the recruitment process are analyzed, comparing differences between those who agreed to be in the study and those who did not. Second, the final sample is compared to the original inference population and different possible subsets, with the goal of determining where the results best generalize (and where they do not). Throughout, these methods are situated in the post hoc analysis of results from two scale-up studies. The article ends with a discussion of the use of these methods more generally when reporting results from randomized trials.

Original languageEnglish (US)
Pages (from-to)209-228
Number of pages20
JournalJournal of Research on Educational Effectiveness
Volume9
DOIs
StatePublished - Oct 3 2016

Keywords

  • external validity
  • generalization
  • recruitment

ASJC Scopus subject areas

  • Education

Fingerprint Dive into the research topics of 'Site Selection in Experiments: An Assessment of Site Recruitment and Generalizability in Two Scale-up Studies'. Together they form a unique fingerprint.

Cite this