TY - JOUR
T1 - The renaissance of field experimentation in evaluating interventions
AU - Shadish, William R.
AU - Cook, Thomas D.
PY - 2009/1/1
Y1 - 2009/1/1
N2 - Most experiments are done in laboratories. However, there is also a theory and practice offield experimentation. It has had its successes and failures over the past four decades but is now increasingly used for answering causal questions. This is true for both randomized and - perhaps more surprisingly - nonrandomized experiments. In this article, we review the history of the use of field experiments, discuss some of the reasons for their current renaissance, and focus the bulk of the article on the particular technical developments that have made this renaissance possible across four kinds of widely used experimental and quasi-experimental designs - randomized experiments, regression discontinuity designs in which those units above a cutoff get one treatment and those below get another, short interrupted time series, and nonrandomized experiments using a nonequivalent comparison group. We focus this review on some of the key technical developments addressing problems that previously stymied accurate effect estimation, the solution of which opens the way for accurate estimation of effects under the often difficult conditions of field implementation - the estimation of treatment effects under partial treatment implementation, the prevention and analysis of attrition, analysis of nested designs, new analytic developments for both regression discontinuity designs and short interrupted time series, and propensity score analysis. We also cover the key empirical evidence showing the conditions under which some nonrandomized experiments may be able to approximate results from randomized experiments.
AB - Most experiments are done in laboratories. However, there is also a theory and practice offield experimentation. It has had its successes and failures over the past four decades but is now increasingly used for answering causal questions. This is true for both randomized and - perhaps more surprisingly - nonrandomized experiments. In this article, we review the history of the use of field experiments, discuss some of the reasons for their current renaissance, and focus the bulk of the article on the particular technical developments that have made this renaissance possible across four kinds of widely used experimental and quasi-experimental designs - randomized experiments, regression discontinuity designs in which those units above a cutoff get one treatment and those below get another, short interrupted time series, and nonrandomized experiments using a nonequivalent comparison group. We focus this review on some of the key technical developments addressing problems that previously stymied accurate effect estimation, the solution of which opens the way for accurate estimation of effects under the often difficult conditions of field implementation - the estimation of treatment effects under partial treatment implementation, the prevention and analysis of attrition, analysis of nested designs, new analytic developments for both regression discontinuity designs and short interrupted time series, and propensity score analysis. We also cover the key empirical evidence showing the conditions under which some nonrandomized experiments may be able to approximate results from randomized experiments.
KW - Experiment
KW - Matching
KW - Propensity scores
KW - Quasi-experiment
KW - Regression discontinuity
KW - Time series
UR - http://www.scopus.com/inward/record.url?scp=60549116512&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=60549116512&partnerID=8YFLogxK
U2 - 10.1146/annurev.psych.60.110707.163544
DO - 10.1146/annurev.psych.60.110707.163544
M3 - Review article
C2 - 18616391
AN - SCOPUS:60549116512
VL - 60
SP - 607
EP - 629
JO - Annual Review of Psychology
JF - Annual Review of Psychology
SN - 0066-4308
ER -