Totality of evidence: using indirect evidence to bridge the gaps

Catrin Tudur Smith, Paula Williamson, Anthony Marson

Background: Indirect comparisons can be valuable in situations where direct comparisons either do not exist, comprise a limited amount of data, or are unlikely to ever be examined in future trials. Individual patient data (IPD) are available from eight systematic reviews of anti-epileptic drug (AED) trials. Each review compared time to event outcomes for individuals randomised to two AEDs of interest. IPD are available for 4496 patients within 18 randomised controlled trials (RCTs) that examine the effect of six different AEDs. Direct pairwise comparisons provide the best level of evidence, but in epilepsy there are gaps in the evidence where direct randomised evidence does not exist or is inconclusive. Indirect comparisons can be used from within the dataset to summarise the totality of the evidence, bridge gaps in the evidence from direct comparisons and hence provide additional clinically relevant information. The combination of direct and indirect comparisons is considered appropriate in this example as identical review protocols and methods were used originally and trials comparing AEDs taken as monotherapy generally involve similar patients. In addition, IPD allows uniformity across trials for outcome and censoring definition.
Objectives: To examine empirical comparisons of direct and indirect evidence, summarise totality of evidence and explore the effect of patient level covariates.
Methods: As individual patient failure time data are available, the stratified Cox proportional hazards model with fixed treatment effects is employed to evaluate indirect comparisons and ultimately combine all data to provide a summary of the total evidence available. Results: Results from direct and indirect comparisons are generally consistent for this example. The hazard ratio (HR) and 95% confidence interval (CI) for the direct comparison between valproate and phenytoin is 0.92(0.64,1.31) with an indirect estimate of 0.83(0.63,1.11). The combination of direct and indirect evidence improves precision for the comparison of interest with a combined estimate of 0.86(0.69,1.08). As expected, the degree of improvement depends on the relative amounts of direct and indirect evidence. An interaction between treatment and age identified in one original review using direct evidence is confirmed using indirect evidence. Estimates of HR and 95% CI are provided for seven pairwise comparisons where direct evidence is not currently available. Conclusions: Since direct and indirect evidence agrees well for comparisons where direct data are available, treatment effect estimates based solely on indirect evidence are considered to reasonably represent underlying patterns in this example. Summarising the total body of evidence provides important clinical results for comparisons that have not been previously undertaken within an RCT setting. The totality of evidence provides the next best level of evidence and could be used to inform sample size calculations in future trials. Furthermore, as the totality of evidence encompasses different trial settings and patient populations these results have potentially increased generalisability. Acknowledgements: We are grateful to the original monotherapy trialists for providing their individual patient data.

Ottawa 2004 O-057