In this author interview, lead authors Nick Meader and Andrew Moriarity tell us about this recently published Cochrane review, how they worked with a patient advisory group, and how they rose to the challenge of this being their first prognostic review, and the first of this type for Cochrane Common Mental Disorders.
Nick Meader: According to the World Health Organization (WHO), depression is the leading cause of disability worldwide and, during the pandemic, depression symptoms have increased in many countries.
We have effective treatments for depression (like antidepressants and cognitive behavioural therapy). So, a substantial number of people will no longer have symptoms of depression. But about half of those who improve will later experience a relapse or recurrence (become unwell again after an initial improvement). For most of these, relapse or recurrence occurs within a year of an improvement in symptoms.
We wanted to investigate whether there are methods to help identify which people are more likely to experience a relapse or recurrence.
Andrew Moriarty: As a GP, I routinely see the personal impacts of relapse and recurrence of depression. We want to provide strategies to prevent relapse for all patients, but in reality there are all sorts of reasons why this might not happen. Reasons might include resource availability, knowledge of health care professionals, or patient preferences, among others. So until we achieve optimal on-going depression management for all, at least if we can accurately and reliably identify those at higher risk, interventions could be more effectively targeted at those who need them most.
Researchers have started to try find ways to more accurately predict relapse and recurrence by developing prognostic models. Prognostic models combine information from several prognostic factors (also known as predictors) to provide individualised risk predictions. In practice, this would usually involve a healthcare professional entering the values for a number of different predictors – a patient’s age or details of their medical history, for example – and being provided with a risk prediction score for that patient as an output. It is essential for these risk predictions to be reliable, as otherwise wrong clinical decisions could be made based on a person’s predicted risk. This review aimed to identify all prognostic models developed to identify relapse or recurrence of depression.
What did you find out?
AM: We identified ten different prognostic models. These models had been developed across different settings (primary care, secondary care and community settings) with a whole range of different predictors included. Some of these were fairly established predictors (for example, history of previous relapse) and some were less common (such as blood test markers or results of psychological assessments). Only four of the models had undergone ‘external validation’, which means testing of their accuracy and reliability by making predictions in new groups of people and settings.
Some of the models appeared to be potentially good at predicting, but ten out of eleven studies had a high risk of bias, meaning that we need to be careful about how much we can trust the results due to concerns of methodology or gaps in information reported. At present, existing models are unlikely to reliably predict an individual’s risk of relapse or recurrence. There is a need for better quality prognostic model studies for relapse and recurrence of depression.
NM: For me, the surprising finding from our review was how little we can conclude from current research. This partly reflects that research is at an early stage, so we found only 11 studies. It’s also the case that methods are rapidly developing. So with these developments future studies will likely help us to be more confident in their findings.
Many of the studies had significant limitations. For example, studies often did not include enough people in their research. Another key problem was that only 4 studies provided an external validation of their prediction model. If you develop a prediction model from a particular group of people it may work very well on predicting relapse for these people. But what we really want to know is if the model works equally well if we use it to predict relapse in depression for a different group of people.
Without external validation, it is very difficult to know how well the model will apply when used in real life settings. This is a common limitation in prognostic modelling studies and we need more of these types of studies in the future.
As a GP, what do you find most interesting about this study?
AM: Most people who seek help for depression are seen in primary care by GPs. Like most GPs, I feel very comfortable discussing options with people with depression and reaching shared decisions about care. The guidance and evidence is reasonably clear, with room for patient choice. The guidance around what should be offered to people once they have improved, to prevent a relapse, is much less clear. Information on how to identify people who are at higher risk is minimal and there are no evidence-based tools in practice to assist with this.
It’s interesting to me that there have been several efforts to develop prediction tools. It seems to me that such a tool would be very useful in primary care to be used by GPs. However, to be used successfully, a prediction tool must be practical and simple to implement in the real world. It would also need to avoid requiring GPs to spend time gathering a significant amount of extra information. Even if we could be more certain about how well the models identified in this review predict relapse, a lot of the predictors included in the models would not be available to GPs. The potential trade-offs between usability and accuracy are critical for me to consider as a GP. The rest of my fellowship is going to explore this further, and I am going to talk to people with a history of depression and GPs about the results.
I understand you worked with a patient advisory group. How were they involved and what did you take away from their involvement?
AM: An excellent patient advisory group has been involved in this project from the early days of applying for funding. They have kept us focussed on asking the right questions and thinking about how best to use the results. The patient advisory group also read and commented on our Plain Language Summary to ensure that the review is accessible to a lay audience and will continue to advise us on how to disseminate the results more widely. An early realisation for me in discussing this with the group was that, while the subject of prognosis and prognostic modelling is quite technical, the real world implications are potentially very tangible and would have a direct effect on patients. The discussion has been lively and the questions raised were: do such models exist? If so, do they work? And if they work, why are they not being used?
This review has not definitively answered the question of whether relapse prediction is possible, but has put us in stronger position to begin to address these questions. Some of the studies suggest it might be possible to develop a reliable tool and report potentially good predictive accuracy, but weaknesses in the methodology mean we can’t fully trust the results. It might be that we can’t predict relapse, in which case we should make sure that we are doing our best to prevent it from occurring. Our patient advisory group will continue to point us in the right direction.
This is one of the first prognosis reviews on this topic (and the first for Cochrane Common Mental Disorders). Methodologically, how did you find working on this?
I feel really lucky to have worked on one of the first Cochrane Prognosis reviews and it was particularly good fun working on the first for the Cochrane Common Mental Disorders group. Nick and I attended the Cochrane Prognosis Reviews course in Utrecht back in 2019, which was great and introduced us to the different way of thinking required for these reviews. We were also fortunate to get lots of help and support from the Cochrane Prognosis Methods Group, and collaborators in the Centre for Prognosis Research at Keele. For example, we have used the novel PROBAST tool for risk of bias assessment, which I think has worked really well in providing an objective way to assess the quality of each study.
Literature searching is trickier than for most interventions reviews as the prognostic model studies tend to be less consistently labelled. However, again there is a huge amount of work going on to improve methodological standards of prognosis research and we have been lucky to have guidance on searching for prognostic modelling studies available to us. I hope our review can be helpful to other authors considering undertaking a Cochrane Prognosis Review.
NM: I enjoyed the challenge of reviewing prognostic models. I’ve learnt a lot from this process – we went on a great course at the University of Utrecht. Applying these skills in our review has been very satisfying.
In many ways the methods are just an extension of those we use in more traditional reviews. But I’ve found it really fun grappling with the challenges of prognostic modelling studies which can be quite different from those faced in clinical trials.
I’ve also enjoyed working with Andrew on this, as he has the mind of a clinician and also a researcher. So as researchers, we often approach the question in a similar way, but we also have complementary skills (he’s a GP and I’m a statistician) so it’s worked well.
What research is needed in the future?
NM: I think this is an exciting area of research where the evidence is beginning to accumulate. There are probably two main areas of need in future studies.
First, I think we are seeing an improvement in methods over time in this area. We’d like to see future studies building on these improvements. For example, by planning to study enough people so that we can draw more confident conclusions about how well the model predicts relapse. We’d also like to see more external validations of these models.
Secondly, there is also a need for further work on identifying what are the key factors for predicting depression relapse. We hope to address some of the research needs in an upcoming study which Andrew is leading on as part of his Doctoral fellowship.
AM: Speaking from a clinical perspective, the important thing for me is that we find a way to improve outcomes and quality of life for patients. Part of that might involve improving the ability to predict relapse. If we find that we can reliably predict relapse, we also have to then be able to do something to prevent it. We’re likely to need to refine and develop existing interventions into something scalable that can be implemented into practice. In addition to the work we have planned attempting to improve risk prediction, Nick and I are going to work together with some of the wider team on a follow-up Cochrane review of interventions for preventing relapse in primary care – I’m looking forward to it!
This review forms part of Andrew’s National Institute for Health Research (NIHR) Doctoral Research Fellowship, which investigates relapse and recurrence of depression. It was a collaboration between colleagues at the University of York, the Hull York Medical School and Keele University.