Inter-Rater Reliability of Novice Linkers Using an Innovative Sequential Iterative Linking Method to Link Prosthetic Outcomes to The International Classification of Functioning, Disability and Health

Authors

  • Leigh Clarke Discipline of Prosthetics and Orthotics, Department of Physiotherapy, Podiatry, Prosthetics and Orthotics, School of Allied Health, Human Services and Sport, La Trobe University, Melbourne, Victoria; The Australian Orthotic Prosthetic Association, Camberwell, Victoria https://orcid.org/0000-0002-4499-1876
  • Emily Ridgewell Discipline of Prosthetics and Orthotics, Department of Physiotherapy, Podiatry, Prosthetics and Orthotics, School of Allied Health, Human Services and Sport, La Trobe University, Melbourne, Victoria https://orcid.org/0000-0001-8775-0258
  • Xia Li Office of Research and Infrastructure, La Trobe University, Melbourne, Victoria, Australia https://orcid.org/0000-0003-1890-5548
  • Michael P. Dillon Discipline of Prosthetics and Orthotics, Department of Physiotherapy, Podiatry, Prosthetics and Orthotics, School of Allied Health, Human Services and Sport, La Trobe University, Melbourne, Victoria https://orcid.org/0000-0003-3614-7787

DOI:

https://doi.org/10.2340/jrm.v55.2409

Keywords:

reliability, agreement, International Classification of Functioning, Disability and Health, ICF, linking, prosthetics, amputation, outcomes

Abstract

Objective: When linking outcomes to the International Classification of Functioning, Disability and Health (ICF), inter-rater reliability is typically assessed at the conclusion of the linking process. This method does not allow for iterative evaluation and adaptations that would improve inter-rater reliability as novices gain experience. This pilot study aims to quantify the inter-rater reliability of novice linkers when using an innovative, sequential, iterative linking method to link prosthetic outcomes to the ICF.
Methods: Across 5 sequential rounds, 2 novices independently linked outcomes to the ICF. A consensus discussion followed each round that informed refinement of the customized ICF linking rules. The inter-rater reliability was calculated for each round using Gwet’s agreement coefficient (AC1).
Results: A total of 1,297 outcomes were linked across 5 rounds. At the end of round 1 inter-rater reliability was high (AC1 = 0.74, 95% confidence interval (95% CI) 0.68–0.80). At the end of round 3, interrater reliability (AC1 = 0.84, 95% CI 0.80–0.88) was significantly improved and marked the point of consistency where further improvements in inter-rater reliability were not statistically significant.
Conclusion: A sequential iterative linking method provides a learning curve that allows novices to achieve high-levels of agreement through consensus discussion and iterative refinement of the customized ICF linking rules.

LAY ABSTRACT
Outcomes are commonly used in healthcare and research to evaluate the effect of an intervention or treatment, such as the effect a prosthesis has on the ability to walk in the community or participate in activities. Cataloguing outcomes using well-established classification systems, such as the International Classification of Functioning, Disability and Health, is important, as it allows outcomes and research to be described using an internationally understood and agreed language. This study aimed to describe an innovative approach to cataloguing outcomes to the ICF, using a method that provides novices with a learning opportunity. In using this innovative method novices were able to catalogue outcomes to the ICF framework with a similar degree of reliability as experts. This will reduce the barriers to novices conducting this type of research in the future.

Downloads

Download data is not yet available.

Downloads

Additional Files

Published

2023-03-09

How to Cite

Clarke, L., Ridgewell, E., Li, X., & Dillon, M. P. (2023). Inter-Rater Reliability of Novice Linkers Using an Innovative Sequential Iterative Linking Method to Link Prosthetic Outcomes to The International Classification of Functioning, Disability and Health. Journal of Rehabilitation Medicine, 55, jrm00373. https://doi.org/10.2340/jrm.v55.2409

Issue

Section

Original Report

Categories