Pros and Cons of Randomized Controlled Trials and Benchmarking Controlled Trials in Rehabilitation: An Academic Debate within the European Academy of Rehabilitation Medicine

Abstract

The European Academy of Rehabilitation Medicine (EARM) held a debate in Hannover, Germany, on 1st of September 2016 on the pros  and cons of randomized controlled trials (RCTs) and observational effectiveness studies (benchmarking controlled trials; BCTs). The  debate involved a chairperson, a person presenting the substance of the debate, an opponent, and a rapporteur. The academicians participated in the discussion. Eight propositions and proposed statements formed the substance of the debate. There was agreement that a study question should be the starting point of an effectiveness study, and not the study method, i.e. RCT or BCT. The term “benchmarking” was questioned: does it mean market-oriented medicine? It was clarified that benchmarking refers to the methodological features of this study design: there must always be a comparison between peers. It was agreed that BCTs might be better than RCTs for use in rehabilitation studies, in which one often needs multi-centred studies, such as in the assessment of the effectiveness of pathways when there is complexity of processes, health systems, organizational issues, structures and facilities; or where interactions between therapists, doctors and patients differ between centres; and when assessing the implementation of rehabilitation. In addition, BCTs may deal with ethical issues, e.g. the acceptability of interventions, more easily than RCTs. Recommendations regarding the different approaches (RCTs or BCTs) should be provided by the scientific rehabilitation societies. Concern over the validity of BCTs was considered justified, as the validity criteria of BCTs cover all those related to RCTs and include the risk of selection bias between treatment arms. Appropriate description of the essentials of the study object, including adequate description of how the interventions were actualized in comparison to the study plan, are essential features for a valid and generalizable study for both RCTs and BCTs. BCTs are necessary to widen the evidence-base of effectiveness in rehabilitation. It was suggested that the rehabilitation field should support the concept of BCTs. It was proposed that education regarding BCTs is indicated, and stakeholders need to be convinced that BCTs are a valid alternative to RCTs. EARM and other physical and rehabilitation medicine (PRM) bodies could advance the use of BCTs for clinical and health policy decision-making.

LAY ABSTRACT
The European Academy of Rehabilitation Medicine (EARM) held a debate on the strengths and limitations of randomized controlled trials (RCTs) and observational effectiveness studies, also known as benchmarking controlled trials (BCTs), in rehabilitation. The main substance of the debate involved eight propositions and four proposed statements. The term “benchmarking” was questioned: does it mean market-oriented medicine? It was clarified that, as benchmarking refers to the features of the study design; there must be comparison between peers. It was agreed that BCTs might be better than RCTs for use in rehabilitation studies: one often needs multi-centred studies and assessment of the effectiveness of pathways; the rehabilitation processes are complex, and health systems and organizational issues are essential; and the essential interactions between therapists, doctors and patients differ between centres. Also, BCTs may deal with ethical issues more efficiently than RCTs. It was recommended that both RCTs and BCTs should be used in rehabilitation research. An essential feature of a valid and generalizable study (for both RCTs and BCTs) is appropriate description of the essentials of the study object. BCTs were considered necessary for widening the evidence-base of effectiveness in rehabilitation, and the rehabilitation field should support the concept of BCTs. It was proposed that education regarding BCTs is indicated, and stakeholders need to be convinced that BCTs are a valid alternative to RCTs. The EARM and other physical and rehabilitation medicine (PRM) bodies should advance the use of BCTs for clinical and health policy decision-making.