The "Defining and determining which quantitative study designs to include in your systematic review of effects of a healthcare intervention" guidance aims to help Cochrane authors consider two key questions:  

1. Should non-randomized studies of interventions (NRSI) be included in your systematic review of the effects of a health care intervention?; and, if so,  
2. What types of NRSI design may help answer review questions about intervention effects?  

It has been developed primarily for systematic reviews of the effects of healthcare interventions in public health. NRSI are defined in the Cochrane Handbook for Systematic Reviews of Interventions as any quantitative study estimating the effects of an intervention (benefit or harm) that does not use randomization to allocate individuals (or groups of individuals) to intervention groups.  

This guidance was developed as part of a collaborative programme of work aimed at delivering practical resources to help authors and editors translate guidance from the Cochrane Handbook for Systematic Reviews of Interventions into practice.

Recommendation

In May 2024 Cochrane endorsed this guidance document as an optional resource for prospective authors following a recommendation from the Methods Executive, which was informed by the relevant Methods Groups, Cochrane's Methods Support and Review Development Team, a Cochrane Review Group representative and the Methods Executive representative on Cochrane's Editorial Board.  

Resources

• Defining and determining which quantitative study designs to include in your systematic review of effects of a healthcare intervention guidance
• Cochrane Handbook of Systematic Reviews of Interventions:
  • Chapter 24, Including non-randomized studies on intervention effects 

INSPECT-SR (INveStigating ProblEmatic Clinical Trials in Systematic Reviews) is a tool for assessing trustworthiness of randomized controlled trials (RCTs) and is intended for systematic reviews including RCTs. The tool was developed in five stages by researchers at the University of Manchester and Cochrane, working with more than 40 institutions worldwide.  

INSPECT-SR guides a review author through a series of up to 21 checks in four domains:  

• post-publication notices, study conduct,  
• governance and transparency,  
• text and figures,  
• and study results.  

INSPECT-SR prompts the reviewer to make a judgement about the trustworthiness of an RCT in relation to each domain, and to make an overall judgement about the trustworthiness of the study based on the domain-level judgements. It also provides a framework to transparently report the reasons for judgements. The tool is intended to be used prior to the risk of bias assessment phase of a review, as it may lead to exclusion of trials if serious concerns are identified. The scope of the tool does not extend to assessments of risk of bias or to conflicts of interest; therefore, it is not intended to replace tool developed for these purposes.  

Recommendation

In October 2025, Cochrane endorsed the INSPECT-SR as an optional tool for authors of systematic reviews of RCTs who wish to assess the trustworthiness of included trials following a recommendation from the Methods Executive, which was informed by the relevant Methods Group, Cochrane's Methods Support, a Cochrane Thematic Group representative and the Methods Executive representative on Cochrane's Editorial Board.  

Resources

• New tool detects problematic trials before they distort evidence
• Wilkinson J, et al. INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials. medRxiv [Preprint]. 2025 Sep 11:2025.09.03.25334905. doi: 10.1101/2025.09.03.25334905. PMID: 40950444; PMCID: PMC12424918.
  • Currently guidance and the accompanying paper are OSF/preprints; therefore, authors citing the paper should clearly indicate the date they accessed it.

The Intervention Synthesis Questions (InSynQ) checklist and guide for developing and reporting the questions addressed in systematic reviews of interventions, consists of 11 items that cover various aspects of the synthesis questions, including the specification of the synthesis questions (including the grouping of interventions, populations and outcomes; specification of comparisons; methodological groupings) and the role of groups in the synthesis (e.g. for pairwise comparisons, structuring text or tables, exploring possible causes of variation in the effects of the intervention, sensitivity analyses).

The InSynQ checklist was developed as part of a collaborative programme of work aimed at delivering practical resources to help authors and editors translate guidance from the Cochrane Handbook for Systematic Reviews of Interventions into practice.

Recommendation  

In May 2023, Cochrane endorsed the InSynQ checklist as optional for prospective authors following a recommendation from the Methods Executive, which was informed by the relevant Methods Group, Cochrane's Methods Support and Review Development Team, a Cochrane Review Group representative and the Methods Executive representative on Cochrane's Editorial Board.  

Resources

• InSynQ official webpage
• Cochrane Handbook of Systematic Reviews of Interventions:  
  • Chapter 3, Defining the criteria for including studies and how they will be grouped for the synthesis
  • Chapter 9, Summarizing study characteristics and preparing for synthesis
• The study centric data management and analysis collection
• Methods Support Web clinics:  
  • Considering scope at the review planning stage 

When performing a meta-analysis, different methods for calculating the summary effect, heterogeneity (between-study) variance and their allied confidence intervals may yield different, or even conflicting, results. The performance of these methods may vary depending on the characteristics of the meta-analysis (e.g. the number and size of the included studies).  

The Wald-type confidence interval and the DerSimonian and Laird (DL) estimator may be misleading, particularly in small meta-analyses. The Cochrane's Statistical Methods Group recommended the following alternative methods to be implemented in RevMan based on the literature, including statistical simulations and empirical evaluations.

• The restricted maximum likelihood (REML) method to estimate the heterogeneity variance.
• Methods to calculate confidence intervals for the heterogeneity variance using the Q-profile statistics method.
• The Hartung and Knapp, Sidik and Jonkman (HKSJ) method for calculating confidence intervals for the summary effect.
• Methods to calculate prediction intervals (i.e. using the t-distribution or the normal distribution)

Recommendation

In September 2022, Cochrane endorsed these new statistical methods for fitting random effects models following a recommendation from the Methods Executive, which was informed by the relevant Methods Group, Cochrane's Methods Support Unit, the Head of Methods and Evidence Synthesis/Deputy Editor-in-Chief, a Cochrane Review Group representative and the Methods Executive representative on Cochrane's Editorial Board.

Updates  

January 2025: The new random-effects methods in RevMan have been implemented. Authors can now use these methods in their protocols, reviews, and updates.

May 2024: The following addendums to the original proposal have been approved to ensure feasibility and consistency among the proposed methods:

• To use the Q-profile method to calculate the confidence interval for the heterogeneity variance due to implementation issues related to the generalized Q-statistic method.
• To use Tau-square to calculate the I-square statistics in order to reflect the estimated heterogeneity variance under DL or REML.

Resources

• Video- lecture presented by Areti Angeliki Veroniki and recorded during the 2022 online training programme for statistical editors and statisticians supporting Cochrane groups
• Cochrane Handbook of Systematic Reviews of Interventions:  
  • Chapter 10.10, Heterogeneity
• Methods Support Unit Web clinic:
  • Introduction to new random-effects methods in RevMan
  • Demonstration of new random-effects methods in RevMan 

The Clinical interpreter tool is an R Shiny application offering an intuitive and clinically grounded approach to interpreting standardized mean differences (SMDs), which are often abstract and challenging for decision-makers and clinicians to contextualize. By translating SMDs into more interpretable metrics – such as scale-specific units – this tool enhances the practical understanding of treatment effects in systematic reviews and meta-analyses. Users can select from predefined outcome categories and commonly used clinical scales, and the tool facilitates evaluation of whether an effect is likely to be clinically meaningful based on minimally important differences for a scale. The tool also supports probabilistic interpretation of clinical relevance through simulation-based probability modeling.

Users should be aware that the accuracy of re-expression depends on availability and validity of minimally important differences and the assumption that the chosen scale is appropriate for the population and outcome in question. The tool is not applicable when no scale-specific minimally important difference exists, or when re-expression across scales is conceptually invalid.

Recommendation

In October 2025, Cochrane determined that the Clinical Interpreter tool could be used as an optional tool for interpreting SMDs following a recommendation from the Methods Executive, which was informed by the relevant Methods Groups, Cochrane's Methods Support, a Cochrane Review Group representative and the Methods Executive representative on Cochrane's Editorial Board.

Resources

• Cochrane Handbook for Systematic Reviews of Interventions, Chapter 15, Section 15.5.3 and 15.5.3.2
• Cochrane Evidence Synthesis and Methods Journal Tutorial: Standardized mean differences in meta-analysis: A tutorial

The CochrAne qualitative Methodological LimitatiOns Tool (CAMELOT) provides authors of qualitative evidence syntheses with a transparent and systematic method to assess methodological limitations of primary qualitative studies. The CAMELOT Development Group, which includes members of the Cochrane Qualitative and Implementation Methods Group, developed the tool in four stages. The tool is evidence-based, specifically for use in Cochrane reviews or GRADE-CERQual.

CAMELOT consists of 12 domains. Four of the domains are Meta domains. These domains encourage review authors to consider study characteristics beyond how the study was carried out, but which inform the study's conduct and design. Eight of the domains are Method domains and encourage review authors to focus on how the study was planned, designed, and/or conducted. Review authors make an assessment by identifying any concerns regarding the methods used in the study and considering the appropriateness of fit between the Meta and Method domains.

Recommendation

In February 2025 Cochrane endorsed CAMELOT as optional for prospective authors following a recommendation from the Methods Executive, which was informed by the relevant Methods Group, Cochrane's Methods Support Unit, the Head of Methods and Evidence Synthesis/Deputy Editor-in-Chief, a Cochrane Review Group representative and the Methods Executive representative on Cochrane's Editorial Board.

Resources

• Cochrane-Campbell Handbook for Qualitative Evidence Synthesis:
  • Chapter 7, Assessing methodological strengths and limitations of qualitative studies
• Qualitative and Implementation Methods Group website 

Data richness and thickness are among several criteria that can be taken into consideration when sampling studies for inclusion for qualitative evidence syntheses (QES). The data richness and thickness assessment tool can ensure that selected studies align with the methodological needs of the selected synthesis approach.  

This tool was developed in a Cochrane context by methodologists in the Cochrane Qualitative and Implementation Methods Group and authors using an evidence-based approach.  

The aim of the tool is to assist authors in systematically and transparently assessing the conceptual richness and contextual thickness of data within primary qualitative studies. The richness tool evaluates the depth and detail of qualitative data in the studies included in a QES, focusing on contextual detail, data complexity, and the insights provided. The thickness tool measures the comprehensiveness and robustness of qualitative analysis, examining how well the analysis is supported by the data, the level of researcher reflexivity, and the coherence of the findings. The tool supports different QES methods. 

Data richness and thickness are among several criteria that can be taken into consideration when sampling studies for inclusion for qualitative evidence syntheses (QES). The data richness and thickness assessment tool can ensure that selected studies align with the methodological needs of the selected synthesis approach.  

This tool was developed in a Cochrane context by methodologists in the Cochrane Qualitative and Implementation Methods Group and authors using an evidence-based approach.  

The aim of the tool is to assist authors in systematically and transparently assessing the conceptual richness and contextual thickness of data within primary qualitative studies. The richness tool evaluates the depth and detail of qualitative data in the studies included in a QES, focusing on contextual detail, data complexity, and the insights provided. The thickness tool measures the comprehensiveness and robustness of qualitative analysis, examining how well the analysis is supported by the data, the level of researcher reflexivity, and the coherence of the findings. The tool supports different QES methods.

Recommendation

In February 2025 Cochrane endorsed the data richness and thickness assessment tool as optional for prospective authors following a recommendation from the Methods Executive, which was informed by the relevant Methods Group, Cochrane's Methods Support Unit, the Head of Methods and Evidence Synthesis/Deputy Editor-in-Chief, a Cochrane Review Group representative and the Methods Executive representative on Cochrane's Editorial Board.

Resources

• Cochrane-Campbell Handbook for Qualitative Evidence Synthesis:
  • Chapters 5, Searching and identifying studies
  • Chapter 6, Selecting studies and sampling
• Qualitative and Implementation Methods Group website 

The interactive Summary of Qualitative Findings (iSoQ) tool is a bespoke, online tool for applying the GRADE-CERQual approach for assessing confidence in the findings of a qualitative evidence synthesis. The iSoQ tool:

• assists review authors in gathering and structuring all data required to make GRADE-CERQual assessments in one place
• displays the information review authors need to make each GRADE-CERQual assessment
• provides practical tips and guidance that compliments the GRADE-CERQual methods.  

The development and maintenance of iSoQ is a collaboration between the Norwegian Institute of Public Health, the Norwegian University of Science and Technology, the Western Norway University of Applied Sciences, Megan Wainwright Consulting, and the Epistemonikos Foundation. Tool development was funded by the Norwegian Institute of Public Health and was undertaken in close collaboration with the GRADE-CERQual Project Group, a subgroup of the GRADE Working group.  

The iSoQ tool is free to use and authors can export Summary of Qualitative Findings tables and Evidence Profile tables to be included in reports and publications.

Currently, integration between iSoQ and RevMan is not yet supported. Review authors who opt to use the iSoQ tool can manually transfer the iSoQ assessment/tables into RevMan and vice versa. Additionally, although the tool enables review authors to publish Summary of Qualitative Findings tables to the iSoQ database, review authors should ensure they adhere to Cochrane’s copyright policies.  

Recommendation

In February 2025 Cochrane endorsed the iSoQ tool as optional for prospective authors following a recommendation from the Methods Executive, which was informed by the relevant Methods Group, Cochrane's Methods Support Unit, the Head of Methods and Evidence Synthesis/Deputy Editor-in-Chief, a Cochrane Review Group representative and the Methods Executive representative on Cochrane's Editorial Board.

Resources

• Learning resources on qualitative evidence synthesis  
• Cochrane-Campbell Handbook for Qualitative Evidence Synthesis:
  • Chapter 13, Assessing confidence in the evidence using the GRADE-CERQual approach
• Learning Live webinars:
  • How the interactive Summary of Qualitative Findings (iSoQ) tool can help you apply GRADE-CERQual in your Qualitative Evidence Synthesis
  • GRADE-CERQual
• Qualitative and Implementation Methods Group website 

The TRANSFER Approach supports review authors in collaborating with decision makers to ensure an informed consideration, from the beginning of the review process, of the transferability of the review findings to the review context.  

In many reviews, the review team only considers transferability (or related concepts such as applicability, indirectness, relevance) of the review findings at the end of the review process and in an ad hoc manner. By considering factors which may influence transferability early in the review process and in collaboration with decision makers, the review team is better able to systematically and transparently assess how these factors may influence the transferability of the review findings to the context specified in the review, or another context. These assessments will also provide transparency to assessing the GRADE ‘indirectness’ domain or the GRADE-CER-Qual ‘relevance’ component. The guidance from this framework can be used in systematic reviews of effectiveness and qualitative evidence syntheses and could be applied to any kind of decision making.  

TRANSFER includes a guide for collaborating to refine/define the review question, a TRANSFER Conversation guide, a TRANSFER Characteristics of context and TRANSFER Guidance for review authors.

Recommendation  

In September 2025, Cochrane endorsed the TRANSFER approach as an optional tool for review authors following a recommendation from the Methods Executive, which was informed by the relevant Methods Group, Cochrane's Acting Head of Methods Support, a Cochrane Review Group representative, and the Methods Executive representative on Cochrane's Editorial Board.  

Resources

• Cochrane-Campbell Handbook for Qualitative Evidence Synthesis, Chapter 13, Assessing confidence in the evidence using the GRADE-CERQual approach
• Qualitative and Implementation Methods Group website 

Within systematic reviews of diagnostic test accuracy, the QUADAS-C tool can assess risk of bias in test comparisons undertaken in comparative accuracy studies (studies that evaluate two or more index tests).  

QUADAS-C is an extension of QUADAS-2 (i.e., it is a set of additional questions to each QUADAS-2 domain): meaning that it should be used together with QUADAS-2. Each QUADAS-C domain asks about QUADAS-2 judgments for each test and additional signalling questions that are specific for comparisons (e.g. whether a fully paired or randomized design was used) to produce a risk of bias judgment for the comparison. Like QUADAS-2, QUADAS-C is a generic tool for all systematic reviews of diagnostic test accuracy but can be tailored to a specific review question.  

Recommendation

From June 2022, Cochrane endorsed QUADAS-C as optional for prospective authors following a recommendation from the Methods Executive, which was informed by the relevant Methods Group, Cochrane's Methods Support Unit, the Head of Methods and Evidence Synthesis/Deputy Editor-in-Chief, a Cochrane Review Group representative and the Methods Executive representative on Cochrane's Editorial Board.  

Resources

• Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy:
  • Chapter 8, Assessing risk of bias and applicability
• Learning resources on diagnostic test accuracy  
• Screening and Diagnostic Tests Methods Group website