TY - JOUR
T1 - Adding Design Elements to Improve Time Series Designs
T2 - No Child Left Behind as an Example of Causal Pattern-Matching
AU - Wong, Manyee
AU - Cook, Thomas D.
AU - Steiner, Peter M.
N1 - Funding Information:
This work was supported by the Institute of Education Sciences’ Grants R305U07003 and R305D100033. Special thanks go to Christopher Jencks, Larry V. Hedges, and Vivian Wong for feedback.
Publisher Copyright:
© 2015, Copyright © Taylor & Francis Group, LLC.
PY - 2015/4/3
Y1 - 2015/4/3
N2 - Abstract:Some form of a short interrupted time series (ITS) is often used to evaluate state and national programs. An ITS design with a single treatment group assumes that the pretest functional form can be validly estimated and extrapolated into the postintervention period where it provides a valid counterfactual. This assumption is problematic. Ambiguous preintervention functional forms are common, as are other factors affecting posttest means and slopes. Using No Child Left Behind as an example, we demonstrate how adding multiple design elements to the basic ITS structure serves to promote causal inference by limiting alternative interpretations. No added design element is perfect by itself, but we argue that they collectively provide a strong causal warrant when the predictions they engender are complex, the results “cohere” with the predictions, and no alternative can fit the same pattern of predictions even if it can fit some of the
AB - Abstract:Some form of a short interrupted time series (ITS) is often used to evaluate state and national programs. An ITS design with a single treatment group assumes that the pretest functional form can be validly estimated and extrapolated into the postintervention period where it provides a valid counterfactual. This assumption is problematic. Ambiguous preintervention functional forms are common, as are other factors affecting posttest means and slopes. Using No Child Left Behind as an example, we demonstrate how adding multiple design elements to the basic ITS structure serves to promote causal inference by limiting alternative interpretations. No added design element is perfect by itself, but we argue that they collectively provide a strong causal warrant when the predictions they engender are complex, the results “cohere” with the predictions, and no alternative can fit the same pattern of predictions even if it can fit some of the
KW - No Child Left Behind
KW - coherence
KW - comparative interrupted time series
KW - pattern matching
UR - http://www.scopus.com/inward/record.url?scp=84926415744&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84926415744&partnerID=8YFLogxK
U2 - 10.1080/19345747.2013.878011
DO - 10.1080/19345747.2013.878011
M3 - Article
AN - SCOPUS:84926415744
VL - 8
SP - 245
EP - 279
JO - Journal of Research on Educational Effectiveness
JF - Journal of Research on Educational Effectiveness
SN - 1934-5747
IS - 2
ER -