Redesigning Implementation Measurement for Monitoring and Quality Improvement in Community Delivery Settings

The field of prevention has established the potential to promote child adjustment across a wide array of outcomes. However, when evidence-based prevention programs have been delivered at scale in community settings, declines in implementation and outcomes have resulted. Maintaining high quality implementation is a critical challenge for the field. We describe steps towards the development of a practical system to monitor and support the high-quality implementation of evidence-based prevention programs in community settings. Research on the implementation of an evidence-based parenting program for divorcing families called the “New Beginnings Program” serves as an illustration of the promise of such a system. As a first step, we describe a multidimensional theoretical model of implementation that links aspects of program delivery with improvements in participant outcomes. We then describe research on the measurement of each of these implementation dimensions and test their relations to intended program outcomes. As a third step, we develop approaches to the assessment of these implementation constructs that are feasible to use in community settings and to establish their reliability and validity. We focus on the application of machine learning algorithms and web-based data collection systems to assess implementation and provide support for high quality delivery and positive outcomes. Examples are presented to demonstrate that valid and reliable measures can be collected using these methods. Finally, we envision how these measures can be used to develop an unobtrusive system to monitor implementation and provide feedback and support in real time to maintain high quality implementation and program outcomes.