(This is a transfer from Columbia University) This project aims to develop a user-friendly webtool that incorporates a new wave of methodological developments on generalizability and moderator analyses with standard methods for study design and power analysis. The goal is to bring these new developments – aimed at increasing the relevance and usefulness of high-quality randomized trials – to applied researchers planning studies. While IES goals and statistical developments have shifted focus beyond the detection of average treatment impact to better understanding moderators of these impacts and the detection of effects for specific inference populations, standard tools aimed at applied researchers have become out-of-date. To that end, we seek to accomplish three primary goals. 1) We will develop a user-friendly webtool that integrates a priori power analysis found in Optimal Design (standard methods) with new methods found in The Generalizer (a webtool aimed at improving generalizations) and methods for moderator analyses (currently only available in PowerUp!). 2) In order to build this tool, we will address gaps in the methodological literature, providing linkages between standard methods for power analysis for the sample average treatment effect (ATE) and methods for detecting a population ATE, as well as power for moderators. 3) In order to better understand current practice with regards to this new focus on relevance and treatment effect heterogeneity – as well as to better inform the development of the tool – we will examine cluster randomized trials (CRTs) funded by IES between 2005 – 2015 in terms of their capacity to generalize to relevant populations and detect variation in treatment effects. The findings from this study will help improve the capacity of education researchers to design CRTs with both high internal and external validity, pushing the field forward into a new era of research with relevance in mind.
|Effective start/end date||5/10/19 → 8/31/22|
- Institute of Education Sciences (R305D180002-19-2)
Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint.