Abstract
Assessment of fidelity that is effective, efficient, and differentiates from usual practices is critical for effectively implementing evidence-based programs for families. This quasi-experiemntal study sought to determine whether observational ratings of fidelity to the Family Check-Up (FCU) could differentiate between levels of clinician training in the model, and from services as usual, and whether rating segments of sessions could be equivalent to rating complete sessions. Coders rated 75 videotaped sessions—complete and 20-min segments—for fidelity, using a valid and reliable rating system across three groups: (a) highly trained in FCU with universal, routine monitoring; (b) minimally trained in FCU with optional, variable monitoring; and (c) services as usual with no training in the FCU. We hypothesized that certain dimensions of fidelity would differ by training, whereas others would not. The results indicated that, as expected, one dimension of fidelity to the FCU, Conceptually accurate to the FCU, was reliably different between the groups (χ 2 = 44.63, p < .001). The differences observed were in the expected direction, showing higher scores for therapists with more training. The rating magnitude of session segments largely did not differ from those of complete session ratings; however, interrater reliabilities were low for the segments. Although observational ratings were shown to be sensitive to the degree of training in the FCU on a unique and theoretically critical dimension, observational coding of complete sessions is resource intensive and limits scalability. Additional work is needed to reduce the burden of assessing fidelity to family-centered programs.
Original language | English (US) |
---|---|
Pages (from-to) | 16-28 |
Number of pages | 13 |
Journal | Journal of Clinical Child and Adolescent Psychology |
Volume | 48 |
Issue number | 1 |
DOIs | |
State | Published - Jan 2 2019 |
Funding
Justin Smith was supported by National Institute on Drug Abuse grant DA027828 (PI Hendricks Brown); National Institute of Mental Health research training grant MH20012 (PI Elizaebeth Stormshak); and by the Implementation Research Institute (IRI) at the George Warren Brown School of Social Work, Washington University in St. Louis through an award from the National Institute of Mental Health (5R25MH08091607) and the Department of Veterans Affairs, Health Services Research & Development Service, Quality Enhancement Research Initiative (PI Enola Proctor). Thomas Dishion, Daniel Shaw, and Melvin Wilson were supported by National Institute on Drug Abuse grant DA016110. Samantha Montag was supported by National Institute on Drug Abuse grant DA027828. Elizabeth Stormshak was supported by Centers for Disease Control and Prevention grant CE001389. The authors thank the members of the research team in Eugene, Phoenix, Pittsburgh, Portland, and Charlottesville, and the families who participated in the two trials. Specifically, we appreciate the contribution of Lisa Brauer for her work in developing the COACH fidelity coding system and coding of videotaped intervention sessions. We are also grateful to C. Hendricks Brown for statistical consultation and Enola Proctor for feedback on early drafts of this paper.
ASJC Scopus subject areas
- Clinical Psychology
- Developmental and Educational Psychology