Quality Assurance Editors in Cochrane Central Editorial Service use the questions below to assess the methods of submitted protocols for intervention reviews. We make these questions freely available so you can check your work before you submit.
This is the checklist for protocols. Go to the methods peer review checklist for reviews and updates.
The assessment questions follow the structure of headings and subheadings in RevMan. For further information on preparing your manuscript for submission, see Check your work meets our standards.
Background
- Is there a clearly defined rationale for why it is important to do this review?
- Are the objectives, eligibility criteria, and PICO clear and logical from the background?
Methods
Types of studies
- Are eligible trial designs stated and justified? (e.g., define quasi-randomised, e.g., "randomised by day of week". Assume that unless otherwise stated, cluster RCT, cross over studies, and multi arms studies are eligible. If not, must be specified and justified here)
Types of participants
- Are participants clearly defined, for example, criteria around location, setting, diagnoses or definition of condition and demographic factors?
- Is there a plan for including studies with a subset of eligible participants?
Types of interventions
- Are interventions clearly defined, including mandatory components?
- Are active and inactive comparators adequately described?
- Are dose, frequency, duration and other aspects of the intervention adequately described?
Outcome measures
- Are the most important outcomes selected (ideally no more than 7) and categorized as critical and important? Do these outcomes include benefits and harms?
- Are outcomes adequately described? E.g. units, scales, timeframes, primary time point.
- Will studies be included regardless of outcomes reported? If excluded based on outcomes reported, is appropriate rationale presented?
Selection of studies and data extraction
- Are two people working independently on selecting studies, extracting data, and performing ROB assessment, with a plan for resolving disagreements, and for contacting trial authors?
- If applicable, authors need to specify details of crowdsourcing or automation tools used in the processes. In the respective sections, authors can briefly include the name of the automation tool and leave the further suggested reporting details to the section ‘Disclosure of artificial intelligence use.’
Risk of bias assessment in included studies
- Is Cochrane RoB1 tool sufficiently described? Do any changes in domain have clear, robust justification?
- Have authors considered assessing objective and subjective outcomes separately?
- If authors plan to use an automation tool to assess risk of bias due to missing results, report how you will use the tool, how it will be trained, and details of its performance and internal validation (name the tool here and include the further details in the section ‘Disclosure of artificial intelligence use.’
- If non-randomized studies are eligible for inclusion, have all ROBINS-I (or similar tool for NRSI studies) guidance points been adhered to?
Additional questions for reviews using Risk of bias 2
- Assessment of risk of bias in included studies
- Have the authors stated the effect of interest (assignment or adherence)?
- Have the authors listed results which will be assessed using RoB2 including outcomes, outcome measures, and time points?
- Cluster RCTs and cross-over RCTs
- Have authors described methods for considering RoB in cluster RCTs and cross-over RCTs as per the interim guidance?
- If you plan to use an automation tool to assess risk of bias due to missing results, report how you will use the tool, how it will be trained, and details of its performance and internal validation (name the tool here and include the further details in the section ‘Disclosure of artificial intelligence use.’
Measures of treatment effect
- Have the authors stated the effect measures that will be used for continuous data (E.g., MD, SMD) and dichotomous data (RR, OR, RD, etc.), and other relevant data types (such as count and rate data) that are referred to in the 'Outcome measures' section?
Unit of analysis
- Are the following considered (if appropriate to review question)?
- Cross-over trials?
- Cluster trials?
- Multiple arm studies?
- Split-body designs (only if applicable to research question)?
Dealing with missing data
- Is there an appropriate plan for missing data?
Reporting biases assessment
- Is there an appropriate plan for assessment of reporting biases? (e.g., use of funnel plots, but only if there are more than 10 studies in an analysis)?
Synthesis methods
- Have authors specified their comparisons of interest in this review? Is the scope of this review ’manageable’? Have authors planned for the smallest number of comparisons that address the main objectives?
- Has fixed / random effects been declared in advance?
- If Fixed Effect Model is to be used, ensure authors have stated the method they will use (such as Mantel-Haenszel, or Inverse-variance)
- If Random Effects Model is to be used;
- Will authors calculate heterogeneity using DL or REML (REML is the recommended approach)?
- Will authors calculate the summary mean confidence interval using Wald or HKSJ (decision will depend on tau2 value and number of studies)?
- Do authors plan to present a confidence interval for the heterogeneity value? (optional!)
- Do authors plan to present a prediction interval? (optional!)
- Have authors described intended methods for synthesizing studies without meta-analysis?
Investigation of heterogeneity and subgroup analyses
- Have authors described how heterogeneity will be assessed? Consider statistical heterogeneity, clinical heterogeneity and methodology heterogeneity?
- Are subgroup analyses pre-defined and justified? (These should relate to the background and PICO and be kept to a minimum (i.e. 3-5)
- Will subgroups be assessed using the formal Test for Subgroup Differences in RevMan?
Equity-related assessment
- Have the review authors considered issues of equity, and planned for appropriate methods to address them if they are?
- If equity related assessment is not to be conducted, is this stated and justified?
Sensitivity analysis
- Are sensitivity analyses clearly defined and appropriate?
Certainty of evidence assessment
- Is there a clear plan to present only the most important comparisons included in the SoF table? Ensure one table per comparison.
- Are the most important outcomes (max 7) to be included in the SoF table pre-specified and consistently worded?
- Are outcomes a balance of benefits and harms?
- Have authors specified which time points and method of measurement will be prioritized for each outcome?
- Are two people working independently to assess the certainty of the body of evidence with a plan for resolving disagreements?
- Are GRADE considerations clearly described and appropriately referenced?
- If you plan to use an automation tool to support the assessment of certainty, report how you will use it, how it was trained, and details of its performance and internal validation (name the tool here and include the further details in the section ‘Disclosure of artificial intelligence use.’
Patient and public involvement
- If patients and/or public are involved, have authors reported specific details and methods (their level of involvement, general approach, roles ,stage and any formal research methods or techniques to be used)?
- Have authors included a supplemental file to describe the planned involvement of patients/public?
- If patients and/or public were not involved, is this stated and justified?
Conflicts of interest
- Are any authors cited in the background section, as this could mean they are authors of included studies?
Supplementary materials
- Are there are details provided in main text that should be a 'supplementary material'?
Disclosure of artificial intelligence use
- If authors have not used AI in (1) the conduct of the review, (2) the preparation of the manuscript, or (3) both, they should use the following statements:
- We did not use AI platforms, systems or tools for any review processes.
- We did not use AI platforms, systems or tools in the preparation of the manuscript.
- We did not use AI platforms, systems or tools for any review process or in the preparation of the manuscript.
- In the disclosure, authors should state 1) the name and purpose of the tool, 2) description of the degree of human oversight, and 3) justify the use of the AI system or tool.
