As part of our commitment to the highest standards of evidence, Cochrane is strengthening how it identifies and manages retracted publications associated with studies included in published or ongoing Cochrane reviews. This includes tagging retracted studies within our CENTRAL database and new steps to manage retractions of included studies that occur after reviews have been published.
Cochrane’s systematic reviews are widely recognized for their independence and integrity and are a trusted source of evidence to inform health-related decision-making. Many clinical and public health guidelines cite or use evidence from Cochrane reviews in formulating recommendations. Maintaining trustworthy evidence requires robust standards both before and after publication.
Prior to publication
As the volume of published research grows, so too does the number of retractions. Retractions occur when there is no longer confidence in the results and conclusions reported in a paper, and so studies associated with these retractions should not be included in systematic reviews (Cochrane policy is that these should be listed as excluded studies).
We have long-standing guidance to identify and exclude retracted studies during review development and editorial processes. Author teams should include a qualified information specialist or librarian, search sources such as Retraction Watch and consider other mechanisms for identifying post-publication amendments, report that these checks have been undertaken, and that studies with retractions have been excluded.
After publication
Systematic reviews reflect the best available evidence at the time they are published. However, when a study included in a review is later retracted, this may affect the reliability of the review’s findings and conclusions.
Publishers of systematic reviews have a responsibility to ensure readers are informed when included studies are retracted and to explain the potential impact on the review. Where a retraction undermines confidence in the conclusions, a decision may be required on whether the systematic review itself should be retracted, in line with guidance from the Committee on Publication Ethics (COPE). This aligns with Cochrane’s position that the purpose of retraction or withdrawal is to correct the literature and ensure its integrity, not to punish the authors.
Recent technological developments have made it easier to identify and manage retractions at scale. The Retraction Watch Database is now openly available through Crossref, making it easier to systematically identify retracted publications.
Cochrane is using these data to tag retracted studies within its CENTRAL database. We are matching these studies to included studies in Cochrane reviews so we can focus on those that have the potential to impact a review’s findings and conclusions.
Determining whether removing an included study (or studies) results in a meaningful change to the review findings or conclusions is not a simple process. To ensure a consistent and transparent approach, we have developed a new framework and accompanying guidance for Cochrane review teams and other researchers to apply to systematic reviews, as well as guidance for editors on deciding whether an editorial note or retraction is required for the published review itself.
Next steps
We’ll be contacting authors of affected reviews and editorial notes will be added to the published reviews to alert readers while the framework is being applied. We encourage authors to update their review as soon as they are able and move studies with retractions from included to excluded; in the interim, editorial notes will ensure transparency for readers. Where retractions are found to meaningfully affect a review’s conclusions, a decision will be made on whether the review requires retraction.
Work is also underway to further automate the tagging of retracted studies in CENTRAL and to develop systems to alert authors of Cochrane reviews when a reference to an included study is newly retracted.
Addressing included studies with retractions is a first step in maintaining the integrity and trust in published Cochrane reviews. Published Cochrane reviews may also include potentially problematic included studies that do not yet have an associated retraction. If that is the case, we recommend that these studies be brought to the attention of the journal that published them so that the editors can investigate the issues in accordance with COPE guidance. In addition, authors are encouraged to complete a full update of their review and apply a tool to assess the trustworthiness of included studies, such as INSPECT-SR.
FAQs for Cochrane authors, readers, and community
Why is Cochrane focusing on retractions of included studies in published reviews now?
The number of retracted publications is increasing. At the same time, improvements in access to retraction data, such as the public availability of the Retraction Watch Database, have made it possible to identify retracted publications more efficiently and systematically.
Cochrane has used these developments to identify retracted publications in its database of controlled trials (CENTRAL) and to determine whether any published Cochrane reviews include these studies. This work allows Cochrane to focus on retractions that may affect review findings, rather than references that appear only in excluded studies or background sections. Future work will further automate the tagging of retracted studies in CENTRAL and support alerts to authors of Cochrane reviews in development when an included study is newly retracted.
How will I know if a published Cochrane review has included studies with associated retractions?
As a first step, an editorial note will be added to affected reviews. These notes inform readers that the review includes a study with an associated retraction and that an assessment of its impact on the review’s findings and conclusions is underway.
I am a Cochrane review author, and my systematic review has included studies with associated retractions. What do I do now?
Cochrane recognizes that authors may wish to take immediate action. As a first step, please apply the framework for determining the impact of a study or studies with an associated retraction and then contact researchintegrity@cochrane.org.
Alternatively, authors will be contacted by Cochrane before the editorial note is posted on their published Cochrane review to inform them their review has included studies with associated retractions , along with more information about how to assess the impact using the above-mentioned framework, and what editorial actions may be taken after this assessment.
Authors will be asked to use the framework for determining the impact of a study or studies with an associated retraction. Once the impact of retracted studies is determined, an editorial decision will be made on whether removing these studies impacts the findings and conclusions. In line with guidance on retractions from the Committee on Publication Ethics (COPE), if the editors have lost confidence in the results and conclusions reported in the paper, the Cochrane review may ultimately require retraction. Otherwise, an updated editorial note will be posted on the published Cochrane review to inform readers that the effect of the included studies with an associated retraction has been assessed, and it has been determined that the findings of the Cochrane review have not been meaningfully impacted, and the conclusions remain reliable.
Independent of the outcome of the assessment, authors will be encouraged to update their review so they can move the affected studies to the excluded studies list.
Why can’t I as an author simply move the retracted included studies to the excluded studies list and republish it?
Cochrane policy states that removing a study should only occur during an update because the removal of study(ies) could have a significant impact on the findings and conclusions, requiring re-analyses, changes to statistical models (e.g., depending on the weight of the study in the analyses) assessments of the certainty of the evidence (e.g., removal could impact of bias or distribution of effect sizes, impacting inconsistency), and updates to text throughout, so an amendment is not appropriate. In addition, the publication date of a Cochrane review showcases when the review was considered up to date. Publishing updates with a new publication date without a new search or using outdated methods would be inappropriate as it would appear to readers that these reviews are ‘up to date’.
As an author, should I treat retractions differently depending on the reason they were retracted?
No. The framework for determining the impact of retracted included studies does not take the reason for retraction into account as publishers will retract if there are grounds for retraction, but may not include all the reasons in the published notice, e.g. due to legal arguments.
Why are there no thresholds for retracting a Cochrane review based on the impact of retracted included studies?
The framework does not specify a definitive threshold because multiple factors may influence the decision to retract, including the contribution of affected studies to the synthesis, changes in the magnitude or direction of effect estimates, shifts in statistical uncertainty, and the impact on GRADE certainty assessments.
During development, there was no consensus on how a threshold should be defined. Views ranged from considering 15% of the data originating from a retracted study sufficient to warrant retraction, to requiring more than 50%, while others considered that retraction of a review would rarely be appropriate solely on this basis. In addition, there are currently no established thresholds for changes in effect size, precision, or certainty of evidence that would automatically trigger retraction. The framework is therefore intended to be flexible and applied on a case-by-case basis, taking into account the specific context of each review. Cochrane is also conducting research to inform whether clearer evidence-based thresholds can be developed in the future.
What should I do if I’m a Cochrane author and already working on an update?
Authors who are already preparing an update should continue their work. If they are informed of a retraction affecting their review during this process, they should: exclude the retracted study, and update the analyses, findings and text throughout accordingly.
Who can authors contact with questions?
Authors with questions about this process or who wish to discuss their specific review can contact Cochrane’s Research Integrity team at: researchintegrity@cochrane.org.
