What certainty can there be on how much treatments Work: empirical insight from recursive cumulative meta-analyses

Ioannidis JPA., Lau J., Clinical Trials and Evidence-Based Medicine Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece and Division of Clinical Care Research, New England Medical Center Hospitals, Tufts University School of Medicine, Boston, USA.

Evidence about how much medical interventions work may change over time. We aimed to determine what fluctuations in the treatment effect reported by clinical trials and their meta-analyses may be expected and whether extreme fluctuations signal future major changes. Methods. We applied recursive cumulative meta-analysis of randomized controlled trials to evaluate the relative change in the pooled treatment effect (odds ratio) over time for 60 medical interventions in 2 medical disciplines (pregnancy/perinatal medicine n=45 interventions; myocardial infarction n=15 interventions). We evaluated the scatter of relative changes for different numbers of total patients in previous trials. Outlier cases were noted with changes > 2.5 standard deviations of the expected. Results. With 500 accumulated patients, the pooled odds ratio may change by 0.6 to 1.7-fold in the immediate future. When 2000 patients have already been randomized, the respective figures are between 0.74 and 1.35-fold for pregnancy/perinatal medicine and between 0.83 and 1.21-fold for myocardial infarction studies. Extreme early fluctuations in the treatment effect were observed in three interventions (magnesium in myocardial infarction, calcium and antiplatelet agents for prevention of pre-eclampsia) where recent mega-trials have contradicted prior meta-analyses as well as in 4 other examples where early large treatment effects were dissipated when more data appeared. Conclusions. Past experience may help quantify the uncertainty surrounding the treatment effects reported in early clinical trials and their meta-analyses. Early wide oscillations in the evolution of the treatment effect for specific interventions may signal further major changes in the future.

Cape Town O20/H4>