Evaluating delivery of a CBT-based group intervention for schoolchildren with emotional problems: Examining the reliability and applicability of a video-based adherence and competence measure

Abstract

Adherence and competence are essential parts of program fidelity and having adequate measures to assess these constructs is important. The Competence and Adherence Scale for Cognitive Behavioral Therapy (CAS CBT) was developed to evaluate the delivery of cognitive therapies for children with clinical anxiety. The present study is an assessment of the slightly adapted version of the CAS CBT evaluating the delivery of a Cognitive Behavioral Therapy (CBT)-based preventive group intervention: EMOTION: <i>Kids Coping with Anxiety and Depression</i>. This study was part of a Norwegian cluster randomized controlled trial (cRCT) investigating the effectiveness of a transdiagnostic intervention, the EMOTION program—an indicated prevention program targeting anxious and depressive symptoms. The applicability and psychometric properties of the CAS CBT were explored. Results are based on six raters evaluating 239 video-recorded sessions of the EMOTION program being delivered by 68 trained group leaders from different municipal services. Interrater reliability (intraclass correlation coefficients, ICC [3, 1]) indicated fair to good agreement between raters. Internal consistency of the instrument's key domains was calculated using the Omega coefficient which ranged between 0.70 to 0.94. There was a strong association between the two scales Adherence and Competence, and inter-item correlations were high across the items, except for the items rating the adherence to the session goals. Competence and Adherence Scale for Cognitive Behavioral Therapy is a brief measure for use in first-line services, with some promising features for easily assessing program fidelity, but some of the results indicated that the instrument should be improved. Future attention should also be made to adapt the instrument to fit better within a group setting, especially regarding evaluation of session goals. More research on how to adequately evaluate fidelity measures are also warranted.