TY - JOUR
T1 - Measuring parent strategy use in early intervention
T2 - Reliability and validity of the Naturalistic Developmental Behavioral Intervention Fidelity Rating Scale across strategy types
AU - Sone, Bailey J.
AU - Kaat, Aaron J.
AU - Roberts, Megan Y.
N1 - Funding Information:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research is part of a trial funded by the National Institutes of Health (NIH) National Institute on Deafness and Other Communication Disorders (NIDCD) (1R01DC014709).
Publisher Copyright:
© The Author(s) 2021.
PY - 2021/10
Y1 - 2021/10
N2 - Children with autism spectrum disorder benefit from early, intensive interventions to improve social communication, and parent-implemented interventions are a feasible, family-centered way to increase treatment dosage. The success of such interventions is dependent on a parent’s ability to implement the strategies with fidelity. However, measurement of parent strategy use varies across studies. Most studies use one of two types of observational coding measures (macro- and micro-codes). Macro-codes are known for being efficient while micro-codes are known for being precise. This study evaluates the reliability and validity of the NDBI-Fi, a macro-code, compared to a micro-code. Parent–child interaction videos for 177 participants were used to compare these measures. Results demonstrated that the NDBI-Fi had strong inter-rater reliability. It also had strong convergent validity with the micro-code after intervention. In addition, the NDBI-Fi was sensitive to change, and it demonstrated precision comparable to the micro-code. Furthermore, a novel scoring procedure detected differences in parents who learned different intervention strategy types. However, the NDBI-Fi did not demonstrate strong validity before intervention, particularly when measuring responsive intervention strategies. Taken together, findings support the use of the NDBI-Fi as an outcome measure, and future work should focus on continued development of valid pre-intervention macro-codes. Lay abstract: Children with autism spectrum disorder benefit from early intervention to improve social communication, and parent-implemented interventions are a feasible and family-centered way to increase the amount of treatment they receive. For these treatments to be effective, it is important for the parent to implement the strategies as intended. However, measurement of parent strategy use is inconsistent across studies of parent-implemented interventions. This study evaluates the quality of the NDBI-Fi, an efficient measure, compared to a more time-consuming measure that is known to be precise. Videos of parents playing with their children were used to compare these two measurement methods. Results demonstrated that the NDBI-Fi was of good quality: scorers had high levels of agreement, the NDBI-Fi was similar to the more precise measure in rating parents after intervention, it detected changes from before to after intervention, and it detected differences when parents learned different types of intervention strategies. The NDBI-Fi was not as precise as the other measure across all strategies before parents learned intervention. Taken together, the findings of this study support the use of the NDBI-Fi as a high-quality outcome measure.
AB - Children with autism spectrum disorder benefit from early, intensive interventions to improve social communication, and parent-implemented interventions are a feasible, family-centered way to increase treatment dosage. The success of such interventions is dependent on a parent’s ability to implement the strategies with fidelity. However, measurement of parent strategy use varies across studies. Most studies use one of two types of observational coding measures (macro- and micro-codes). Macro-codes are known for being efficient while micro-codes are known for being precise. This study evaluates the reliability and validity of the NDBI-Fi, a macro-code, compared to a micro-code. Parent–child interaction videos for 177 participants were used to compare these measures. Results demonstrated that the NDBI-Fi had strong inter-rater reliability. It also had strong convergent validity with the micro-code after intervention. In addition, the NDBI-Fi was sensitive to change, and it demonstrated precision comparable to the micro-code. Furthermore, a novel scoring procedure detected differences in parents who learned different intervention strategy types. However, the NDBI-Fi did not demonstrate strong validity before intervention, particularly when measuring responsive intervention strategies. Taken together, findings support the use of the NDBI-Fi as an outcome measure, and future work should focus on continued development of valid pre-intervention macro-codes. Lay abstract: Children with autism spectrum disorder benefit from early intervention to improve social communication, and parent-implemented interventions are a feasible and family-centered way to increase the amount of treatment they receive. For these treatments to be effective, it is important for the parent to implement the strategies as intended. However, measurement of parent strategy use is inconsistent across studies of parent-implemented interventions. This study evaluates the quality of the NDBI-Fi, an efficient measure, compared to a more time-consuming measure that is known to be precise. Videos of parents playing with their children were used to compare these two measurement methods. Results demonstrated that the NDBI-Fi was of good quality: scorers had high levels of agreement, the NDBI-Fi was similar to the more precise measure in rating parents after intervention, it detected changes from before to after intervention, and it detected differences when parents learned different types of intervention strategies. The NDBI-Fi was not as precise as the other measure across all strategies before parents learned intervention. Taken together, the findings of this study support the use of the NDBI-Fi as a high-quality outcome measure.
KW - autism spectrum disorders
KW - communication and language
KW - fidelity
KW - observational methods
KW - outcome measure
KW - parent-implemented intervention
UR - http://www.scopus.com/inward/record.url?scp=85106479186&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85106479186&partnerID=8YFLogxK
U2 - 10.1177/13623613211015003
DO - 10.1177/13623613211015003
M3 - Article
C2 - 34030519
AN - SCOPUS:85106479186
VL - 25
SP - 2101
EP - 2111
JO - Autism
JF - Autism
SN - 1362-3613
IS - 7
ER -