Ensuring intervention fidelity in rehabilitation research

Keeping Current. © B. Di Rezze, 2012

Why is measuring intervention fidelity important?

Pediatric rehabilitation interventions are tailored for the child and family based on theories of treatment, and the way in which these concepts are operationalized can create variation in delivery. As a result, it can be challenging to make generalizations about the impact of rehabilitation interventions from a research study. One way to address this issue is to define interventions in a detailed treatment manual (manual-based treatment) in order to standardize key characteristics of therapist treatments.1 Methods of evaluating manual-based treatments, such as measuring a therapist's fidelity to the intervention, are essential to improving the rigor of rehabilitation research.2,3

Fidelity to treatment or intervention fidelity refers to the degree to which an intervention or program is delivered as intended.4 Recent evidence has indicated that intervention fidelity should be considered within Consolidated Standards of Reporting Trials (CONSORT) to be part of randomized controlled trials (RCTs) in non-medical study designs.5 In the literature, intervention fidelity is synonymous with such terms as treatment or implementation fidelity, program fidelity, therapist/clinician's adherence or competence, treatment integrity and procedural reliability. By evaluating therapist's fidelity to treatment, the internal and external validity of an intervention approach could be established to know how well the treatment was conducted within a study, and how well it can be generalized to other clinical settings. Moreover, the use of such a measure can improve the ease with which rehabilitation therapists can apply evidence to pediatric practice, because the fidelity measures will describe the essential characteristics of treatment delivery conducted in the study.

What do we know about how intervention fidelity is measured?

To date, the best work in intervention fidelity methodology and measurement has occurred within psychology research. Researchers in this field have developed and tested methods to address the challenges in measuring their complex treatment practices and to establish methodological standards for intervention fidelity.6

Traditionally, the theoretical basis of measuring intervention fidelity comes from work by Waltz and colleagues (1993) who emphasized the evaluation of therapist behaviours.7 In this conceptual model, therapist's adherence and competence with the intervention are the two areas evaluated. Adherence addresses the extent to which a therapist uses interventions and approaches detailed by the treatment manual and competence refers to the level of skill shown by the therapist in delivering the treatment.7 For this theory, the need to evaluate both components of intervention fidelity is essential to accurately assess the therapeutic components that influence changes in a child's abilities and participation. Various measures have been established in the psychology literature to evaluate a therapist's fidelity for manual-based treatments, such as Cognitive-Behavioural Therapy in the Yale Adherence and Competency Scale (YACS).8

More recently, a group of researchers in the U.S. from the National Institute of Health - Behavior Change Consortium (NIH-BCC)9 based fidelity evaluation on a conceptual model by Lichstein and colleagues (1994).10 This model considers the evaluation of therapist behaviours as well as additional processes involving the client. This model considers therapist delivery of treatment, the ability for the client to understand how to use the learned skills and how well the client can apply skills.10 In addition, the NIH-BCC model identifies characteristics of fidelity measurement to be incorporated in the intervention study (e.g., study design and training for intervention providers).

A third conceptual model, the Implementation Fidelity Framework (IFF) (See Figure 1) includes concepts that integrate characteristics of fidelity within an intervention session by examining both client involvement (referred to as responsiveness) and therapist performance, including adherence and quality of delivery.4 Therapist adherence mediates between an intervention plan and fidelity evaluation, and is different from potential moderators (e.g., client responsiveness and quality of therapy delivery) that may increase or decrease the level of fidelity achieved. This model also takes the larger study components into consideration (i.e., facilitation strategies). The framework recognizes that all of these factors can influence fidelity of intervention.

How is intervention fidelity currently measured in rehabilitation interventions?

Evaluation of implementation fidelity in non-medical research has recently become more prevalent. Nursing has adopted treatment fidelity methods based on the concepts by Waltz and colleagues (1993) to increase the trustworthiness of their interventions. In the rehabilitation literature, recent descriptive papers have discussed the importance of evaluating implementation fidelity; however, there is a scarcity of methods to measure it.3 Previous fidelity measures that have been utilized in rehabilitation research have not been based on conceptual models of fidelity and have largely evaluated components of intervention at a program level (e.g., whether or not intervention was conducted) rather than measuring the specific characteristics of an intervention. Since therapy in rehabilitation is equally complex and dynamic in its treatment practices, the need to develop and implement methods to measure intervention fidelity is essential. Using a conceptual model as a foundation for fidelity measurement will ensure that the key features of the intervention are understood, delivered and examined appropriately within an intervention study.

What concepts of intervention fidelity are important for paediatric rehabilitation?

The evaluation of intervention fidelity in rehabilitation has only recently explored the measurement of therapist behaviours when conducting specifically defined interventions. One direction that has yet to be examined in rehabilitation is the evaluation of both therapist and client behaviours within a treatment session. Nelson and Mathiowetz (2004) reported that the guidelines published by the NIH-BCC are a good fit for rehabilitation research. In paediatric rehabilitation, the success of implementing an intervention is often contingent on the dynamic interactions between the therapist and the child or family. A concept that is not covered by the NIH-BCC model is the examination of the skillfulness or the quality of the therapy delivered.8 In paediatric rehabilitation, it is important to measure the quality of the therapy because of the clinical considerations that need to be integrated with the intervention itself, such as a child's developmental level, level of ability, and interests.11,12

Thus, an ideal model for rehabilitation could incorporate the performance behaviours of the therapist, the child and the parent (if applicable), as well as the concept of quality of the therapy delivered. The Implementation Fidelity Framework4 is comprehensive in describing these concepts to guide the evaluation of fidelity in a paediatric rehabilitation session. Within this framework, the key factors potentially influencing fidelity include the dynamic nature of intervention involving the child and/or parent (i.e., client responsiveness) and the therapist's performance (i.e., adherence and quality of therapy delivery).

Where do we go from here?

Using the IFF4 as a guide, a measure of intervention fidelity for pediatric rehabilitation is being developed. The key concepts from this model forming the basis of this measure will be related to therapist adherence, client responsiveness and quality of therapy delivery. This measure is being developed through CanChild as an adjunct to a study called, "Family centred functional therapy for children with cerebral palsy" (referred to as the Focus on Function Study).13,14

The primary aim of the Focus on Function study is to conduct a multi-site RCT to evaluate and compare the efficacy of two approaches to paediatric rehabilitation intervention. These treatments aim to improve performance of functional tasks and mobility, increase participation in everyday activities, and improve quality of life in children with cerebral palsy. Within the study, occupational therapists and physiotherapists provide treatment using one of two manual-based rehabilitation approaches which both focus on improving functional performance: (i) Child-focused approach - changing the child's skills and abilities; and (ii) Context-focused approach - changing the task or environment supporting performance.

The fidelity measure developed in this study will identify and examine the consistency and validity of observable frequency and quality of the therapist behaviours for each intervention within the Focus on Function study. In addition, client and client-therapist interactions will be examined to better understand their potential moderating behaviours on intervention fidelity.

What is the potential impact of this fidelity measure in paediatric rehabilitation?

The intervention fidelity project will develop a measure to evaluate the therapist's and client's behaviours within each intervention of the Focus on Function study. Even though the fidelity measure will be developed within the context of the Focus on Function study, its principles, the methods of its design and the use of the measure as an adaptable scale have the potential to be useful for other areas of paediatric rehabilitation intervention. Therefore, this research can lead to important improvements in the quality of paediatric rehabilitation intervention research across disciplines. This work is also potentially useful for clinicians to measure therapist performance for the purpose of therapist training in particular treatment methods to increase quality of practice. Most importantly, it is hoped that the application of this knowledge within clinical rehabilitation practice will ultimately provide children and their families with a better quality of health care.

Want to know more about intervention fidelity or the project at CanChild?

Contact Briano Di Rezze at direzzbm@mcmaster.ca. Briano is a Post-doctoral fellow at the Offord Centre of Child Studies and the CanChild Centre for Childhood Disability Research. The focus of his PhD research was in developing a generic measure of intervention fidelity in paediatric rehabilitation. This measure will be utilized within the aforementioned 'Focus on Function Study' being conducted through CanChild and aims to be a framework for other interventions for children with disabilities. 

  • Click here for list of references

    Whyte, J., & Hart, T. (2003). It's more than a black box; It's a Russian doll: Defining rehabilitation treatments. Amer J of Phys Med Rehab, 82, 639-652.

    Dijkers, M., Kroop, G. C., Esper, R. M., Yavuzer, G., Cullen, N., & Bakdalieh, Y. (2002). Quality of intervention research reporting in medical rehabilitation journals. Amer J Phys Med, 81, 21-33.

    Nelson, D. L., & Mathiowetz, V. (2004). Randomized controlled trials to investigate occupational therapy research questions. Amer J Occup Ther, 58, 24-34.

    Carroll, C., Patterson, M., Wood, S., Booth, A., Rick, A., & Balain, S. (2007). A conceptual framework for implementation fidelity. Implementation Science, 2(40). DOI:10.1186/1748-5908-2-40.

    Davidson, K. W., Goldstein, M., Kaplan, R. M., Kaufmann, P. G., Knatterud, G. L., Orleans, C. T., Spring, B., Trudeau, K. J., & Whitlock, E. P. (2003). Evidence-based behavioral medicine: What is it and how do we achieve it? Ann Behav Med, 26, 161-171.

    Bond, G. R., Evans, L., Salyers, M. P., Williams, J., & Kim, H. W. (2000). Measurement of fidelity in psychiatric rehabilitation. Mental Health Services Research, 2, 75-87.

    Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993). Testing the integrity of psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620-630.

    Carroll, K. M., Nich, C., Sifry, R. L., Nuro, K. F., Frankforter, T. L., & Ball, S. A. (2000). A general system for evaluating therapist adherence and competence in psycho-therapy research in the addictions. Drug and Alcohol Dependence, 57, 225-238.

    Bellg, A. J., Borrelli, B., Resnick, B., Hecht, J., Sharp Minicucci, D., Ory, M., Ogedegbe, G., Orwig, D., Ernst, D., & Czajkowski, S. (2004). Enhancing treatment fidelity in health behaviour change studies: Best practices and recommendations from the NIH Behaviour Change Consortium. Health Psych, 23, 443-451.

    Lichstein, K. L., Riedel, B., & Grieve, R. (1994). Fair tests of clinical trials: A treatment implementation model. Advances in Behaviour Research & Therapy, 16, 1-29.

    Dunst, C. J. (2009). Implications of Evidence-Based Practices for Personnel Preparation Development in Early Childhood Intervention. Infants and Young Children, 22, 44-53.

    Rodger, S., & Keen, D. (2010). Child- and Family-Centred Service Provision. In S. Rodger (Ed.), Occupation-centred Practice with children (45-69), West Sussex, UK: John Wiley & Sons, Inc.

    Law, M., Darrah, J., Pollock, N., Wilson, B., Russell, D., Walter, S., Rosenbaum, P., & Galuppi, B. (2011). Focus on function: a cluster, randomized controlled trial comparing child- versus context-focused intervention for young children with cerebral palsy. Developmental Medicine and Child Neurology, 53, 621-629.

    Darrah, J., Law, M. C., Pollock, N., Wilson, B., Russell, D. J., Walter, S. D., Rosenbaum, P., & Galuppi, B. (2011). Context therapy: a new intervention approach for children with cerebral palsy. Developmental Medicine & Child Neurology, 53, 615-620.