Accepted Apr 29, 2021; Epub ahead of print May 7, 2021

J Rehabil Med 2021; 53: jrm00207

Correspondence address: Boya Nugraha, Department of Rehabilitation Medicine, Hannover Medical School, Carl-Neuberg-Straße 1, D-30625 Hanover, Germany. E-mail: boya.nugraha@gmail.com

Doi: 10.2340/16501977-2842

Many rehabilitation studies show contradictory results. A possible reason for this is the difference in rehabilitation settings (rehabilitation service organizations). This paper describes the need for a checklist to report these factors for rehabilitation studies. It will be derived based on the International Classification of Service Organization in Rehabilitation (ICSO-R) 2.0. In addition, this will include not only a systematic review, but also a Delphi-based survey and consensus workshop.

The number of clinical trials in the field of rehabilitation medicine is increasing. However, in many cases the results in terms of treatment effects are contradictory. This is one of the reasons why many meta-analyses still conclude that evidence from clinical trials is weak, and more trials “with good methodology” are needed. A possible reason for this could be that effects of the settings (or service organization) have not been considered, even though they clearly could influence the study outcomes. Some examples for such factors are: payment or refunding for interventions; the professions working in the rehabilitation team; the involvement of the user in the decision-making process; and the documentation and quality assurance system used. Other aspects could be whether the service focuses mainly on rehabilitation or on curative treatment, or whether services are delivered mainly in the outpatient, day-clinic or inpatient setting. Such factors should be described in rehabilitation trials and evaluated as confounding factors on the effects of interventions.

The influence of factors related to the organization of rehabilitation services on decision-making have recently been conceptually described by Gutenbrunner & Nugraha (2). Such influence is particularly relevant in rehabilitation, as improvements in functioning are the main goals of rehabilitation interventions. Outcomes of rehabilitation can therefore only be understood if the interaction of an individual with a health condition and the environment is taken into account. In this context rehabilitation service organization can be considered as a relevant contextual factor. However, a systematic review of randomized controlled trials (RCTs) of rehabilitation services for patients with traumatic brain injury (TBI) showed that these factors are very rarely reported (3).

For the standardized reporting of relevant factors in parallel-group randomized trials, the CONSORT (Consolidated Statement of Reporting Trials) group has published a statement that is widely accepted and applicable to clinical trials (4, 5). However, for trials in rehabilitation some specific conditions, e.g. related to complex interventions, qualification of service providers, background treatments, and service organization, are not reflected. This, for example, concerns the fact that many interventions cannot be applied with blinding of the patients/participants (6), that interventions are mostly multimodal and/or complex (7), and that the interventions are dependent on contextual factors or even include such factors in the rehabilitation concept (8, 9). To better describe non-pharmacological interventions, Hoffmann et al. (10) proposed an extending reporting standard entitled: “Template for Intervention Description and Replication” (TIDieR). This checklist includes dimensions on the provider (type of health professional) and location of service delivery, as well as some aspects of quality control. Another approach is the “Consensus of Exercise Reporting Template” (CERT) published by Slade et al. (11). Here, too, aspects related to the qualifications of professionals applying the treatment and the location of service delivery are addressed. However, neither of these approaches used systematic screening of relevant service-related factors, but focused on “immediate” factors related to service delivery.

One of the reasons why service organization factors are not considered in publications on clinical trials could be a lack of a commonly agreed framework to describe these factors. In 2015, Gutenbrunner et al. (12) published an initial proposal on the dimensions and categories relevant to describe service organization in rehabilitation. After testing and revision, an updated version of this International Classification of Service Organization in Rehabilitation (ICSO-R) 2.0 was published recently (1). This updated version can now be used to systematically describe rehabilitation settings; however, the classification is too extensive to be reported in each clinical trial. Hence, the aim is to develop a brief set of factors that should be used in clinical trials in the field of rehabilitation medicine. The development of such minimum reporting should be based on the results of literature review and a consensus process of experts (i.e. Delphi rounds and workshops).

This paper provides relevant evidence and perspectives, and describes a methodological approach for a project to develop a minimum reporting set of relevant factors related to service organization for clinical trials in rehabilitation.

The influence of factors related to service organization is demonstrated by the following examples:

• A study in which aspects of service organization in rehabilitation became obvious was a RCT on the effects of a comprehensive rehabilitation programme on follow-up of an inpatient rehabilitation in women with breast cancer (13). The intervention comprised a set of rehabilitation interventions consisting of physiotherapy (including manual lymphatic drainage), occupational therapy, hydrotherapy, medical training and cognitive behavioural treatments. The control was treatment as usual. Whereas the complex rehabilitation treatments were offered in the outpatient department of rehabilitation located in a university hospital, treatment as usual was delivered by family doctors, comprising prescription of drugs and physiotherapy alone. Randomization was performed after obtaining informed consent at the end of the inpatient rehabilitation phase. Unexpectedly, the drop-out rate in the intervention group was significantly higher, as in the control group (13). Thus, the study failed, as the outcomes could not be evaluated statistically since the number of cases in the intervention group was too low. For better understanding of the reasons behind this phenomenon, those patients who gave consent to be followed up by phone were asked about their motivation. The most common answer was that they did not want to go to a university hospital as they did not feel severely ill and that family members and neighbours would ask if they still had cancer. This was seen as a disruption of the concept that the cancer had been overcome. It could be expected that the drop-out rate might have been lower if the treatment had been delivered in the community. From the perspective of the ICSO-R this phenomenon is related to categories 2.5 (location of service delivery) and 2.7 (setting).
• Another study describes the barriers and facilitators to utilization of rehabilitation services. Although this study is not a RCT, it reflects the importance of considering factors related to the health system and organization in clinical trials. This study aimed at determining barriers and facilitators to accessing rehabilitation services amongst persons with lower-limb amputations in a rural community in South Africa (14). It found that distance and accessibility were barriers to utilization of rehabilitation services. Meanwhile, supported referral system and positive experiences with rehabilitation professionals were facilitators for the utilization of rehabilitation services. In summary, these barriers and facilitators, which are described in the ICSO-R 2.0, can influence not only the process of accessing rehabilitation services, but also the outcome of utilizing rehabilitation services. In ICSO-R 2.0, rural area and distance are related to 2.5 (location of service delivery), accessibility is related to 2.6 (facility), referral is in category 2.4 (mode of referral), and health profession is related to 2.11 (rehabilitation team).
• Further, a quasi-experimental study (15) investigated the effect of integrating rehabilitation into acute care for severe TBI and direct transfer to a specialized rehabilitation hospital (the early rehabilitation and continuous chain of rehabilitation) compared with a broken chain of rehabilitation that starts in the sub-acute phase (the delayed rehabilitation group). It became clear that the aspects of rehabilitation service organization influenced outcomes of this study; a better functional outcome occurs in patients who receive early-onset and a continuous chain of rehabilitation. From the perspective of ICSO-R this phenomenon is related to categories 1.1 (context), 2.5 (location of service delivery), 2.7 (setting), 2.8 (integration of care), 2.8 (patient–centeredness), 2.10 (aspects of time and intensity), 2.11 (rehabilitation team) and 2.12 (reporting and documentation).
• In the scientific literature, some trials have shown that aspects of service organization influence the outcomes of clinical trials in rehabilitation:
• Doig et al. (16) showed that rehabilitation outcomes are highly dependent on the setting in which the service is delivered (ICSO-R 2.0 category 2.7);
• Medina-Mirapeix et al. (9) demonstrated that environmental attributes (ICSO-R 2.0: 1.) strongly influence the perception of the quality of a rehabilitation service;
• the influence of patient involvement (ICSO-R 2.0 categories 2.7–2.9) with long-term outcomes of rehabilitation of neurological diseases were shown by Ashford et al. (17);
• Longley et al. (18) showed that clinical decision-making in clinical rehabilitation is, among others, influenced by organizational pressure, which can be seen as a factor in governance (ICSO-R 2.0 category 1.4);
• Al-Rashaida et al. (19) in a trial on vocational rehabilitation demonstrated that service factors, such as counsellor characteristics (ICSO-R 2.0 category 2.11), the characteristics of vocational rehabilitation services (ICSO-R 2.0 category 2.7), and other centre features are important for outcomes and satisfaction of users;
• In addition, Malmivaara (20, 21) recently proposed:

◦ The need for benchmarking RCT, based on systematic reviews that show the importance of including factors for RCT in order to reduce a poor/misleading conclusion from RCTs. In addition to patients’ characteristics, this benchmarking includes some service organization factors, such as human resources (ICSO-R 2.0 category 1.6), staff competencies (ICSO-R 2.0 category 2.11.1), and catchment area (ICSO-R 2.0 category 2.5.2) (20, 21).

◦ That systematic reviews should also report the (population, intervention, control, and outcomes) (PICO) characteristics of the RCTs in sufficient detail; both in the study protocol, and the actual characteristics in practice in the study. The suggested characteristics include some factors related to service organization, as mentioned above.

However, these descriptions should only be seen as examples, and more systematic research is required on this topic.

From a theoretical point of view, aspects of service organization may influence all phases of a clinical trial, and are therefore relevant for outcomes (see Fig. 1). These factors may relate to the provider or the service delivery. The idea of a study is, for example, biased by whether the (larger) institution is an academic body, a profit or non-profit organization, and by service delivery aspects, such as service goals, patient groups, etc. In addition to the effect of the institutional background, the research plan is influenced by the rehabilitation team available, the technical equipment, and the treatment capacities. For ethical committee approval the institution (including financial aspects) is important. Patient recruitment is again dependent on service delivery aspects, such as the spectrum of disease and functioning parameters, accessibility, and the rehabilitation team. For evaluation, the information and technology (IT) structure and the documentation methods as well as the expertise of team members are relevant. Finally, yet important for publication, the financial factors and goals of the institution are of importance and, of course, implementation depends on many factors related to the service provider and service delivery.

Andelic et al. (22) published a topical review analysing rehabilitation service descriptions in recently completed rehabilitation RCTs based on the ICSO-R 2.0. The study included 29 RCTs published in 2018. Studies with a wide range of organizational units and target groups were included. A median of 4 (range 3–5) categories were reported in the provider dimension. In the service delivery dimension the median was 8 (range 6–12) categories. This shows that only a very few dimensions related to service organization are reported in clinical trials. The authors of this review concluded that a standard should be developed to report such factors. This study found that, in clinical trials, factors related to rehabilitation service organization neither are not systematically reported, and the influence of these factors on outcomes has not been conclusively investigated. This situation highlights the need for a systematic approach to developing reporting standards of service organization for use in clinical trials.

Approach to systematically describing service organization in rehabilitation

Rehabilitation services have been described conceptually as “personal and non-personal intangible products offered to persons with a health condition experiencing or likely to experience disability, addressing individual functioning needs, and delivered by rehabilitation professionals, other health professionals, or appropriately trained community-based workers” (23). To describe such services, it is necessary to distinguish between the provider and the service delivery domain. The provider domain is defined as “organizational units with the primary goal to provide rehabilitation services” (1) and can be interpreted as “hardware” for the service. The service delivery is an “offer of a set of products (interventions, procedures, devices, pharmaceuticals and other goods, etc.) to a specified group of persons (patients, informal caregivers and/or other users or clients) aiming at achieving or maintaining optimal functioning (rehabilitation) within an organizational context (provider)” (1) and can be seen as “software”. Both domains contain a separate category of funding, as payment for the organization itself may differ from payment for the services or service delivery, respectively.

All dimensions, categories and subcategories of the ICSO-R 2.0 (Box 1) have short definitions and are described by inclusion and exclusion criteria (1). Those descriptions and criteria have been added to make clear what is meant by the single term and will contribute to the feasibility of use and the precision of answers. However, there is still a lack of defined measures or scales for quantifying ICSO-R 2.0 categories. A project to develop so-called value-sets has been inaugurated within the ICSO-R working group of the World Health Organization (WHO)-Liaison Committee of the International Society of Physical and Rehabilitation Medicine (ISPRM).

Methodological approach to developing a minimum reporting set for relevant factors of service organization for rehabilitation trials

In conclusion, the examples presented in this paper indicate that factors related to service organization are relevant to the outcomes of clinical trials in rehabilitation medicine. However, it is not yet clear which factors are most relevant to be reported when publishing the results of such clinical trials. In order to improve the standardization of reporting of domains of service organization, a minimum reporting set should be developed based on further research and a systematic consensus process among international experts in rehabilitation sciences. The ICSO-R 2.0 can be used as the basis for such work. Therefore, this paper provides a methodological approach (Table I) for use in developing a minimum reporting set for relevant factors related to service organization for rehabilitation study. It includes 2 systematic reviews, Delphi exercises, focus group discussion and consensus workshops. The first systematic review is to identify rehabilitation service descriptions in rehabilitation RCTs by using the ICSO-R 2.0 (22). The second systematic review is to analyse how the rehabilitation service organization factors (based on ICSO-R 2.0) could influence the results of the rehabilitation studies. The Delphi exercise aim is to identify the most important service organization factors (based on ICSO-R 2.0) in rehabilitation study. For the Delphi survey, international experts with broad expertise in rehabilitation are invited. Finally, all of the results will be discussed in a consensus conference, which will include experts in rehabilitation fields. The resulting set of categories should be aligned with commonly used standards, such as the CONSORT (5) or the RCTs in Rehabilitation Checklist (RCTRACK) approach (24).

The authors have no conflict of interest to declare.