ORIGINAL REPORT
Boya NUGRAHA, MS, PhD1, Grace ENGEN, MSC2, Cecilie ROE, MD, PhD2,3,4, Marit KIRKEVOLD, RN, EDD, PhD2, Helene L. SOBERG, PT, PhD2, Nada ANDELIC, MD, PhD2,3 and Christoph GUTENBRUNNER, MD, PhD1
From the 1Department of Rehabilitation Medicine, Hannover Medical School, 30625-Hannover, Germany, 2Research Centre for Habilitation and Rehabilitation Models and Services (CHARM), Institute of Health and Society, Faculty of Medicine, 3Department of Physical Medicine and Rehabilitation and 4Institute of Clinical Medicine, Faculty of Medicine, University of Oslo, Oslo, Norway
Objective: To identify the most important categories of the International Classification of Service Organization in Rehabilitation (ICSO-R 2.0) for a minimum reporting data set.
Methods: A 2-step Delphi survey was used. Rehabilitation experts from all world regions including physicians, nurses, neuropsychologists, physiotherapists, and others, were invited to participate. In the first round, all participants were asked to rate the categories and subcategories of the ICSO-R 2.0 with the following criteria: Being relevant for study outcomes; Being distinctive among different rehabilitation settings; Being feasible to use and reported by objective figures or other clear characterization. All categories that were rated relevant, distinctive and feasible by more than 60% of respondents from the first round were included in the second round.
Results: The most important and relevant factors for the minimum reporting set in rehabilitation services regarding the provider were: human resources, context, technical resources, quality assurance and management, location of provider, and ownership. Regarding the service delivery, the most important and relevant factors were: target group, rehabilitation team, aspect of time and intensity, setting, location of service delivery, modes of referral, facility and reporting and documentation.
Conclusion: Several categories were identified, and reduction in these through discussions and iterative voting at workshops and consensus conferences is needed before finalizing the reporting set.
A goal of this Delphi study was to identify the most important parameters of the International Classification of Service Organization in Rehabilitation (ICSO-R 2.0), to characterize rehabilitation services and identify important missing categories needed in a minimum reporting set. The most important and relevant factors for developing a minimum reporting set regarding the provider were: human resources, context, technical resources, quality assurance and management, location of provider, and ownership. Regarding service delivery the most important and relevant factors were: target group, rehabilitation team, aspect of time and intensity, setting, location of service delivery, modes of referral, facility and reporting and documentation. These factors should be discussed further, and a final set should be developed in workshops through discussion and iterative voting.
Key words: rehabilitation; health service; clinical trial; Delphi study; contextual factor; International Classification System for Service Organization in Health-Related Rehabilitation.
Citation: J Rehabil Med 2022; 54: jrm00265. DOI: https://dx.doi.org/10.2340/jrm.v54.2033.
Copyright: © Published by Medical Journals Sweden, on behalf of the Foundation for Rehabilitation Information. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (https://creativecommons.org/licenses/by-nc/4.0/)
Accepted: Jan 21, 2022; Epub ahead of print: Feb 17, 2022; Published: May 11, 2022
Correspondence address: Boya Nugraha, Rehabilitation Medicine, Hannover Medical School, DE-30625 Hannover, Germany. E-mail: boya.nugraha@gmail.com
Competing interests and funding: The authors have no conflicts of interest to declare.
Rehabilitation has emerged as an important health strategy in different healthcare settings, and a health intervention for different health conditions. There is consensus that rehabilitation is needed not only for post-surgical care, congenital diseases, for people living in cancer, in palliative care, and treatment of non-communicable diseases, but also for sequelae of severe injuries, diseases (e.g. metabolic syndrome, stroke, etc.) and infectious diseases, including COVID-19 (both acute and long-term care) (1, 2). The implementation of rehabilitation interventions, such as other health strategies and health interventions, should be evidence based; in other words, they should be based on the results of clinical trials.
Clinical trials in rehabilitation have emerged during the last 3 decades, due to its relevance for almost every health condition, such as cancer (3–6), stroke (7–10), traumatic brain injury and other neurological disorders (11–13), COVID-19 (14–16), musculoskeletal pain (17, 18), and diabetes (19). Studies on rehabilitation services have been performed in different care settings, including hospitals and community (20). However, many studies using similar rehabilitation interventions have shown different outcomes (21). One of the reasons could be due to external factors, such as the context of the studies (20, 22); for example, the studies assessing rehabilitation services in the university hospital may show different outcomes than those performed in the other care settings (21). For example, a comprehensive follow-up rehabilitation programme after inpatient rehabilitation for women with breast cancer (23) has shown a higher drop-out rate in the intervention group compared with the control group. Questioning the reasons directly to the participants, it is clear that some participants did not feel ill enough to be treated in a university hospital and others faced stigmatism from family and neighbourhood if continuously going to tertiary care hospital. Therefore, the dropout rate could be minimized if the study was delivered in the community or at home (23). This issue refers to the ICSO-R 2.0 location of service delivery and setting. Another study aiming at evaluating barriers and facilitators in utilizing rehabilitation services in South Africa (24) has shown that contextual factors, such as location of service delivery, facility, mode of referral, and health profession, can be barriers or facilitators of the process of accessing rehabilitation services and its outcomes.
These types of factors are not reported in most clinical trials, and, if they are, they are most often not counted as relevant or confounding factors for the outcomes (20).
Evidence-based decision-making in medicine, including in rehabilitation, should be supported by 3 factors: clinical judgement, relevant scientific evidence, and patients’ values, goals and preferences. Gutenbrunner & Nugraha (25) have proposed additional factors to be considered in evidence-based decision-making, which are called health system and service organization factors. The proposed factor can be formulated by using the International Classification of Service Organization in Rehabilitation (ICSO-R; (26)). Recently, Gutenbrunner et al. (27) published the second version of the ICSO-R (ICSO-R 2.0). The ICSO-R 2.0 was used as the framework for screening the external factors that had been reported in rehabilitation studies, in a systematic review by Andelic et al. (20) and Roe et al. (21).
Following these publications, the need to develop a consensus on which external factors, based on ICSO-R 2.0, are important and relevant for reporting in rehabilitation studies emerged. Therefore, a Delphi study was conducted for this purpose (22). The main aim was to identify the most important parameter categories of ICSO-R 2.0, to characterize rehabilitation services. This Delphi study was one of studies of a larger project to develop a minimum reporting set for rehabilitation studies.
ICSO-R 2.0 was used as the framework to identify the most important external factors for rehabilitation trials (27). ICSO-R 2.0 consists of 2 dimensions and 23 categories (see Box 1).
1. Provider 1.1 Context 1.2 Ownership 1.3 Location of provider 1.4 Governance/leadership 1.4.1 Mission 1.4.2 Vision 1.4.3 Involvement in governance and management 1.5 Quality assurance and management 1.6 Human resources 1.7 Technical resources 1.8 Funding of provider 1.8.1 Source of money 1.8.2 Criteria of funding 1.9 Other categories of provider 2. Service delivery 2.1 Health strategies 2.2 Service goals 2.3 Target groups 2.3.1 Health condition groups |
2.3.2 Functioning groups 2.3.3 Other target groups 2.4 Mode of referral 2.5 Location of service delivery 2.5.1 Location characteristics 2.5.2 Catchment area 2.6 Facility 2.7 Setting 2.7.1 Levels of care 2.7.2 Mode of service delivery 2.7.3 Phase of healthcare 2.8 Integration of care 2.9 Patient-centeredness 2.10 Aspect of time and intensity 2.11 Rehabilitation team 2.11.1 Professions, competencies 2.11.2 Interaction approaches 2.12 Reporting and documentation 2.13 Funding of service delivery 2.13.1 Source of money 2.13.2 Critria of payment 2.14 Other categories of service delivery |
All the dimension, categories and sub-categories, including their definitions, and inclusion and exclusion criteria of the ICSO-R 2.0, were gathered into the online data collection platform Nettskjema (University Information Technology Center (USIT), University of Oslo, Norway).
The 2-step Delphi-exercise was performed by experts with a background in clinical rehabilitation as well as in rehabilitation research. The experts had different professional backgrounds, came from all world regions, and worked in different rehabilitation settings. Prior to inviting the experts in the rehabilitation field, international organizations, such as the International Society of Physical and Rehabilitation Medicine (ISPRM), World Physiotherapy, World Federation of Occupational Therapy (WFOT), International Society of Prosthetics and Orthotics (ISPO), International Federation of Social Work (IFSW), World Association of Speech and Language Pathology (WASLP), International Council of Nurse (ICN) and International Neuropsychology Society (INS), were asked to nominate experts to participate in the study. In addition, experts from different regions of the world were also identified by our group. Finally, the invitation for the Delphi study was sent to 96 experts for both the first and the second round.
An invitation e-mail, containing a URL link to the survey, was sent to the identified potential participants, followed by 1 email reminder (2 weeks after the initial invitation). The first round of the survey was performed from 4 May to 1 June 2020;
All participants were asked to rate (as yes/no/cannot decide) the dimensions, categories and sub-categories of the ICSO-R 2.0, along with the following criteria:
All the categories that had been rated relevant, distinctive and feasible by more than 60% of the respondents from the first round were included in the second round. This resulted in 6 out of 9 categories at the provider and 8 out of 14 categories at the service delivery of the ICSO-R 2.0.
The second-round survey was performed from 26 October to 18 November 2020. In this round the 6 categories from the provider and the 8 categories from the service delivery should be ranked. Each category for the provider could be ranked only once (on a scale of 1–6, with 1 being the most important; and 6 the least important to be included in the minimum reporting set for rehabilitation studies). Similarly, the selected 8 categories for service delivery had to be ranked by the participants (on a scale of 1–8; with 1 being the most important; and 8 the least important to be included in the minimum reporting set for rehabilitation studies.
The mean value of each category (based on the rankings by the participants) was computed to obtain the final ranking/priority for both the provider and service delivery of the minimum reporting set for rehabilitation studies.
Participants. Thirty-two out of 96 of the invitees (33%) responded to the survey. The participants were from different regions, but most were Europeans (25%), South and South-East Asians (21.9%), followed by Central and East Asia (18.8%), North America (12.5%), and Middle and South America (9.4%). Middle East, Africa and Oceania represented 12.4% of respondents.
The participants in this survey had diverse professions: physical and rehabilitation medicine (PRM) physicians 68.8%; physiotherapists (PT) 9.4%; neuropsychologist 3.1%; neurologist 3.1%; speech and language therapist (SLT) 3.15%; and others (rehabilitation-related public health workers, prosthetist and orthotist, rehabilitation engineer, health economist) 12.5%.
Provider. Table I shows the result from the first round of Delphi survey on the Provider of the ICSO-R 2.0. All of the categories and subcategories domains of the ICSO-R 2.0 were selected by all participants. The domains that had been selected as being relevant, distinctive and feasible by ≥ 60% of participants would proceed to the second round (see bold text in Table I). The 7 domains for the provider are: context (1.1), ownership (1.2), location of provider (1.3), quality assurance and management (1.4), human resources (1.6), and technical resources (1.7). The overview of all the results including the domains (categories and subcategories) that were rated as being not relevant, not distinctive, not feasible, and “cannot decide” are shown in Table SI.
Service delivery. Table II shows the result from the first round of the Delphi survey on the service delivery of the ICSO-R 2.0. All of the category and subcategory domains of the ICSO-R 2.0 were selected by all participants. The domains that were selected as being relevant, distinctive and feasible by equal or more than 60% of the participants were included in the second round (see bold text and numbers). The 8 domains for the categories are: target group (2.3), modes of referral (2.4), location of service delivery (2.5), facility (2.6), setting (2.7), aspect of time and intensity (2.10), rehabilitation team (2.11), and reporting and documentation (2.12). The overview of all the results, including the domains (categories and subcategories) that were rated as not being relevant, not distinctive and not feasible; and “cannot decide” are shown in Table SII.
In the second round, the survey was sent to 96 invitees. Thirty-two people responded in the second round (33%). However, 7 responses were excluded, as the respondents had not ranked the variables properly (same ranking was given to 2 or more categories). Therefore, only 25 responses were valid and were used to calculate the mean values of the rankings. As shown in Table III, the rankings/priorities were defined based on the mean value of the rankings given by the respondents for each category. From the calculations, the order of importance from the most to the least is as follows for the provider: human resources (1.6), context (1.1), technical resources (1.7), quality assurance and management (1.5), location of provider (1.3) and ownership (1.2).
Table IV shows the rankings for service delivery. Based on the mean rank values, the order of importance from the most to the least is as follows: target group (2.6), rehabilitation team (2.11), aspect of time and intensity (2.10), setting (2.7), location of service delivery (2.5), modes of referral (2.4), facility (2.6), and reporting and documentation (2.12).
The relevance and importance of developing a minimum reporting set for factors relevant to service organization for rehabilitation trials have been discussed earlier in our previous article (22). It has also set out a methodological approach and processes for that purpose. One of these is Delphi study, which is the present study. This study was performed to identify the most important categories of the ICSO-R 2.0 in order to characterize rehabilitation services and identify important missing categories needed in a minimum reporting set in rehabilitation studies. The results will be used as one of the parameters for a consensus meeting to develop a rehabilitation reporting set for rehabilitation studies. Previous studies have been reported as part of the projects (20), including the introduction to the project (22).
The response rate of the first round of the Delphi process was 33% (32/96). One of the barriers to obtaining higher response rates in this study could be the difficulty in understanding the ICSO-R 2.0, despite our having presented the definitions, inclusions and exclusions with surveys. This was compensated by supplementary explanations by the authors and 1 of the ICSO-R developers (BN) to the participants who needed clarification, either by email or by teleconference. As the current study was performed anonymously, it was not possible to identify who had responded on the first round. Therefore, the same set of invitees (n = 96) were invited to participate in the second round without distinguishing the responders from the non-responders. Sixty percent was used as the cut-off point for consensus on the first round, which was in agreement with the suggested cut-off for the Delphi study (28).
Based on this result, some categories have been ranked. The first 3 from provider and first 4 from service delivery can be elaborated as follows:
The results of this study are relevant to the development of reporting criteria for clinical trials concerning factors related to service organization (22). This is based on the grounded hypothesis that rehabilitation outcomes may be influenced by factors such as the service location, structure and profile of rehabilitation team, technical resources and other factors related to service organization (6, 24, 34–38). Such factors can be seen as contextual factors in service delivery, which is in line with the comprehensive World Health Organization (WHO) model of functioning and health (39). It may also reflect the extension of the Sackett’s model of evidence-based decision-making in medicine, for which, in the context of health-related rehabilitation, a fourth factor has been proposed (25).
In a systematic review of clinical rehabilitation trials of disorders of the nervous system, Roe et al. (12) reported that contextual factors related to service provision and delivery were scarcely described. Using the ICSO-R framework, Andelic et al. (20) investigated the reporting of categories of service organization in rehabilitation outcome studies, and demonstrated that this framework of classification is feasible for the systematic reporting of contextual factors in rehabilitation services at the meso level. However, in another systematic review (personal communication with Cecilie Roe). Identified only a few studies that systematically investigated the influence of characteristics pertaining to service organization on rehabilitation outcomes. It was shown that rehabilitation outcomes could be influenced by the setting (particularly, the mode of service delivery), aspect of time, intensity and rehabilitation team.
In order to document the quality of controlled trials in medicine, the use of standards for reporting potentially influential factors is a precondition to include studies in meta-analysis. One of the best known is the statement of Consolidated Standards of Reporting Trials (CONSORT) (31). Factors related to service organization are not included in this reference list or in its specifications. However, the Template for Intervention Description and Replication (TIDieR) checklist and guide for interventional studies (40) included 4 dimensions related to the organization of services. The questions are: “who provided” (“for each category of intervention provider (such as psychologist, nursing assistant), describe their expertise, background, and any specific training given”), “how” (“describe the modes of delivery (such as face to face or by some other mechanism, such as internet or telephone) of the intervention and whether it was provided individually or in a group”), “where” (“describe the type(s) of location(s) where the intervention occurred, including any necessary infrastructure or relevant features”), “when and how much” (“describe the number of times the intervention was delivered and over what period of time including the number of sessions, their schedule, and their duration, intensity or dose”). Last, but not least, Cochrane rehabilitation is developing an extension for the CONSORT statement, in order to provide standards of reporting appropriate for rehabilitation intervention and, in particular, for multidimensional interventions typically used for rehabilitation, which is called the Randomized Controlled Trials Rehabilitation Checklist (RCTRACK) (41).
The strength of this study is its inclusion of a diverse group of participants, both geographically and in terms of their expertise in the field of rehabilitation. This diversity has enriched the findings and reduced the influence of bias on the results. Despite the fact that all types of rehabilitation professionals were identified and invited to participate in this study, an occupational therapist (OT) was not represented in the first round of the survey. In the second round, the participants were not asked to provide information regarding their profession and work location. ICSO-R 2.0 was used as the framework for selection, as it has been developed to describe rehabilitation service organization and systematically report the external factors or settings associated with rehabilitation interventions and studies.
A reporting set of service organization may be useful for the development of a feasible method to characterize the service organization surrounding rehabilitation studies and the management of practical issues in the field of rehabilitation. The results of this study may contribute to the development of such a minimum reporting set. However, additional steps are necessary. The identified categories should be discussed and prioritized in multi-professional working groups, including using the methodological approach that has been developed in the ICF Core-Set projects (42). One important criterion will be the question of whether standardized methods are available to assess the respective category (so-called value-sets) (27). Last, but no least, testing of feasibility, validity and reliability should be performed. Furthermore, the resulting set of categories should be aligned with commonly used standards, such as the CONSORT (31) or RCTRACK approach (41).
As aforementioned, some participants might have difficulty understanding the ICSO-R 2.0. However, the authors responded promptly to questions posed by the participants by email and teleconference. Although all types of rehabilitation professionals had been identified and invited to participate in this study, no OTs, psychologists or general practitioners responded.
This study determined the most important and relevant factors for developing a minimum reporting set in rehabilitation. Regarding the provider, these factors are: human resources, context, technical resources, quality assurance and management, location of provider, and ownership; and, regarding service delivery, the factors are: target group, rehabilitation team, aspect of time and intensity, setting, location of service delivery, modes of referral, facility and reporting and documentation. These factors should be discussed in multi-professional workshops in which consensus can be reached through iterative voting, in order to arrive at a feasible and suitable final reporting set.