Generalizability of findings from systematic reviews and meta-analyses in the Leading General Medical Journals

Abstract

Objective: To assess how items relevant for the assessment of the generalizability of findings from randomized controlled trials were recorded in systematic reviews published in leading general medical journals. Methods: All systematic reviews and meta-analyses published in the Annals of Internal Medicine, BMJ, JAMA (The Journal of the American Medical Association) and Lancet from 1 January 2016 to 28 February 2019 were searched via PubMed. Reporting of the characteristics of randomized controlled trials in the systematic reviews was documented by the benchmarking method. Results: A total of 115 systematic reviews were found. Of these, 71% included pharmacological interventions, 35% included other conservative treatments, 13% included surgical interventions, and 0% included rehabilitation interventions. None of the systematic reviews assessed patient selection, 35% reported disorder-specific clinical features, 25 % reported comorbid conditions, and 21% reported patients? behavioural factors in randomized controlled trials. Functioning, environmental factors and inequity-related factors were recorded in 3%, 0% and 9%, respectively, of the systematic reviews; and adherence to interventions, crossovers, and co-interventions in 7%, 0% and 2%, respectively; follow-up percentages in 8%; and adequacy of statistical analyses in 3%. Conclusion: In all systematic reviews the recording of characteristics of patients, adherence to interventions, follow-up, and statistical analyses in the RCTs was insufficient. The data did not allow assessment of the clinical homogeneity of the randomized controlled trials, or provide justification for meta-analysis, or allow generalizability of the findings.