This systematic review assesses the effects of different control interventions in randomised trials including patients with a mental health disorder. In randomised trials, patients are assigned by chance to one of two or more groups – usually an experimental intervention and a control intervention. There are many types of control interventions in mental health intervention research. Some of the most common are different types of placebos that lack what is assumed to be the active component in the experimental intervention, and usual care, where patients receive the standard treatment for their mental health disorder in the area where they live. Two other types of control interventions are wait-list or no-treatment where patients receive no trial-related care during the study (although some patients may receive care outside the studies). Wait-list patients are often offered the experimental intervention after the trial has been finalised if it is likely to provide more benefits than harms, while no-treatment participants are not offered the experimental intervention by the researchers.
We searched for randomised trials with patients with mental health disorders where wait-list, usual care, or placebo interventions were compared with either wait-list or no-treatment. We looked at differences between all the types of control interventions on beneficial effects and whether they caused any adverse effects. We included 96 trials with a total of 4200 participants. Only 83 trials (3614 participants) provided usable data. Fifteen different mental health disorders were included. We found that all the trials were at high risk of bias in how they had been conducted, which reduced the interpretability of our findings. However, the risk of bias was mostly due to lack of blinding in the placebo studies, which may be seen as an aspect of the review's methodological question rather than a flaw with the review itself. We found no clinically important differences for usual care or wait-list control interventions in the main analyses, however in our secondary analyses we found a clinically important favourable difference for usual care. In general, placebo control interventions tended to be favourable over no-treatment or wait-list control interventions across mental health disorders. We found no clinically important differences on adverse events.
This review suggests that different control interventions have a tendency to yield very different estimates for the effects of the experimental intervention and that the choice of control intervention has a large impact on how effective a mental health treatment appears to be. Control interventions in trials with patients with mental health disorders are often poorly reported upon, and guidelines are needed to inform researchers on how to properly design, report, and interpret these trials.
We found marked variations in effects between placebo versus no-treatment and wait-list and between subtypes of placebo with the same comparisons. Almost all the trials were small with considerable methodological and clinical variability in factors such as mental health population, contents of the included control interventions, and outcome domains. All trials were assessed as high risk of bias and the evidence quality was low to very low.
When researchers decide to use placebos or usual care control interventions in trials with people with mental health disorders it will often lead to lower estimated effects of the experimental intervention than when using wait-list or no-treatment controls. The choice of a control intervention therefore has considerable impact on how effective a mental health treatment appears to be. Methodological guideline development is needed to reach a consensus on future standards for the design and reporting of control interventions in mental health intervention research.
Control interventions in randomised trials provide a frame of reference for the experimental interventions and enable estimations of causality. In the case of randomised trials assessing patients with mental health disorders, many different control interventions are used, and the choice of control intervention may have considerable impact on the estimated effects of the treatments being evaluated.
To assess the benefits and harms of typical control interventions in randomised trials with patients with mental health disorders. The difference in effects between control interventions translates directly to the impact a control group has on the estimated effect of an experimental intervention. We aimed primarily to assess the difference in effects between (i) wait-list versus no-treatment, (ii) usual care versus wait-list or no-treatment, and (iii) placebo interventions (all placebo interventions combined or psychological, pharmacological, and physical placebos individually) versus wait-list or no-treatment. Wait-list patients are offered the experimental intervention by the researchers after the trial has been finalised if it offers more benefits than harms, while no-treatment participants are not offered the experimental intervention by the researchers.
In March 2018, we searched MEDLINE, PsycInfo, Embase, CENTRAL, and seven other databases and six trials registers.
We included randomised trials assessing patients with a mental health disorder that compared wait-list, usual care, or placebo interventions with wait-list or no-treatment .
Titles, abstracts, and full texts were reviewed for eligibility. Review authors independently extracted data and assessed risk of bias using Cochrane’s risk of bias tool. GRADE was used to assess the quality of the evidence. We contacted researchers working in the field to ask for data from additional published and unpublished trials.
A pre-planned decision hierarchy was used to select one benefit and one harm outcome from each trial. For the assessment of benefits, we summarised continuous data as standardised mean differences (SMDs) and dichotomous data as risk ratios (RRs). We used risk differences (RDs) for the assessment of adverse events. We used random-effects models for all statistical analyses. We used subgroup analysis to explore potential causes for heterogeneity (e.g. type of placebo) and sensitivity analyses to explore the robustness of the primary analyses (e.g. fixed-effect model).
We included 96 randomised trials (4200 participants), ranging from 8 to 393 participants in each trial. 83 trials (3614 participants) provided usable data. The trials included 15 different mental health disorders, the most common being anxiety (25 trials), depression (16 trials), and sleep-wake disorders (11 trials).
All 96 trials were assessed as high risk of bias partly because of the inability to blind participants and personnel in trials with two control interventions. The quality of evidence was rated low to very low, mostly due to risk of bias, imprecision in estimates, and heterogeneity.
Only one trial compared wait-list versus no-treatment directly but the authors were not able to provide us with any usable data on the comparison.
Five trials compared usual care versus wait-list or no-treatment and found a SMD –0.33 (95% CI -0.83 to 0.16, I² = 86%, 523 participants) on benefits.
The difference between all placebo interventions combined versus wait-list or no-treatment was SMD −0.37 (95% CI −0.49 to −0.25, I² = 41%, 65 trials, 2446 participants) on benefits. There was evidence of some asymmetry in the funnel plot (Egger’s test P value of 0.087). Almost all the trials were small. Subgroup analysis found a moderate effect in favour of psychological placebos SMD -0.49 (95% CI −0.64 to −0.30; I² = 53%, 39 trials, 1656 participants). The effect of pharmacological placebos versus wait-list or no-treatment on benefits was SMD -0.14 (95% CI −0.39 to 0.11, 9 trials, 279 participants) and the effect of physical placebos was SMD −0.21 (95% CI −0.35 to −0.08, I² = 0%, 17 trials, 896 participants). We found large variations in effect sizes in the psychological and pharmacological placebo comparisons. For specific mental health disorders, we found significant differences in favour of all placebos for sleep-wake disorders, major depressive disorder, and anxiety disorders, but the analyses were imprecise due to sparse data.
We found no significant differences in harms for any of the comparisons but the analyses suffered from sparse data.
When using a fixed-effect model in a sensitivity analysis on the comparison for usual care versus wait-list and no-treatment, the results were significant with an SMD of –0.46 (95 % CI –0.64 to –0.28). We reported an alternative risk of bias model where we excluded the blinding domains seeing how issues with blinding may be seen as part of the review investigation itself. However, this did not markedly change the overall risk of bias profile as most of the trials still included one or more unclear bias domains.