The Sequential Scale-Up of an Evidence-Based Intervention: A Case Study

Jaime Thomas*, Thomas D Cook, Alice Klein, Prentice Starkey, Lydia DeFlorio

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

Policy makers face dilemmas when choosing a policy, program, or practice to implement. Researchers in education, public health, and other fields have proposed a sequential approach to identifying interventions worthy of broader adoption, involving pilot, efficacy, effectiveness, and scale-up studies. In this article, we examine a scale-up of an early math intervention to the state level, using a cluster randomized controlled trial. The intervention, Pre-K Mathematics, has produced robust positive effects on children’s math ability in prior pilot, efficacy, and effectiveness studies. In the current study, we ask if it remains effective at a larger scale in a heterogeneous collection of pre-K programs that plausibly represent all low-income families with a child of pre-K age who live in California. We find that Pre-K Mathematics remains effective at the state level, with positive and statistically significant effects (effect size on the Early Childhood Longitudinal Study, Birth Cohort Mathematics Assessment =.30, p <.01). In addition, we develop a framework of the dimensions of scale-up to explain why effect sizes might decrease as scale increases. Using this framework, we compare the causal estimates from the present study to those from earlier, smaller studies. Consistent with our framework, we find that effect sizes have decreased over time. We conclude with a discussion of the implications of our study for how we think about the external validity of causal relationships.

Original languageEnglish (US)
Pages (from-to)318-357
Number of pages40
JournalEvaluation Review
Volume42
Issue number3
DOIs
StatePublished - Jun 1 2018

Keywords

  • content area
  • design and evaluation of programs and policies
  • education
  • outcome evaluation (other than economic evaluation)
  • program design and development
  • program implementation

ASJC Scopus subject areas

  • Arts and Humanities (miscellaneous)
  • Social Sciences(all)

Fingerprint Dive into the research topics of 'The Sequential Scale-Up of an Evidence-Based Intervention: A Case Study'. Together they form a unique fingerprint.

Cite this