Cochrane & systematic reviews

The main purpose of The Cochrane Collaboration is to develop systematic reviews of the strongest evidence available about healthcare interventions. Consumers and health practitioners can then work together to make the best possible decisions about health care. The reviews are published electronically within The Cochrane Library and are freely accessible in shortened versions (abstracts and consumer summaries).

Why Cochrane?

The reality is that healthcare providers face a serious challenge to keep up-to-date with the latest health care. Research is being published every day, all over the world. Information appears in thousands of medical, scientific and health-related journals worldwide. Furthermore, the results of one healthcare study may be different from, or even contradict, the results of another study making it difficult to draw conclusions. Journals are published in a variety of languages and Cochrane reviewer authors attempt to use translated articles, wherever possible, to produce a review.

What is The Cochrane Collaboration?

The Collaboration is a not-for-profit organisation based in the UK. To carry out its mission of producing systematic reviews, a central administration (the Secretariat) supports disease and health condition based groups (collaborative review groups) in developing the systematic reviews. The review groups are spread across the world, generally based at universities and teaching hospitals.
The Cochrane Collaboration involves more than 10,000 people, worldwide. Most do not receive a payment for their Cochrane work as they are committed to its purpose. You can find out more about the history of the Collaboration, which began in the area of pregnancy and childbirth in 1993.

The main activity of the Collaboration is the preparation of Cochrane reviews that are published electronically in successive issues of The Cochrane Library. By publishing electronically, reviews can be added to the Library and existing reviews brought up-to-date as new clinical studies are completed and reported.

The Cochrane Collaboration is named after a UK doctor, Archie Cochrane. Dr Cochrane studied patterns of disease and how these relate to the environment and life patterns; he was an epidemiologist. In the early 1970’s, Dr Cochrane aroused a lot of interest by pointing out that many decisions about health care are made without reliable, up-to-date evidence about the treatments used.

People are still surprised to hear that many healthcare decisions are made without strong evidence.

A Cochrane consumer shares his early impressions

As a biochemist, retired in 1995 after 45 years at the laboratory bench, I had been looking for a new scientific interest. I have atopic eczema and joined the National Eczema Society in 1996. Through exchange, I became aware of the Cochrane Skin Group and attended one of their Consumer Forums. This led to my becoming involved in handsearching dermatology journals for them.

At a meeting I learned of Archie Cochrane, a Professor of Epidemiology at Cardiff, who envisaged a library containing the results, analysis and conclusions of every controlled clinical trial ever carried out. Ideally, the treatments should be randomly allocated to patient groups and neither the administering physician nor the patient should know what treatment is given. Only by such stringent controls can clinical trials be performed in a truly objective manner. The first clinical trial reported was in 1948.

Doctors base their treatments on what they have been taught, their experience and what they have gleaned from the medical literature and from pharmaceutical companies. This information may be out-of-date, biased or incomplete. Doctors are able to spend only a small amount of time each week reading journals (there are currently about 100 dermatology journals). A systematic review of a group of clinical trials abstracted from journals showed that new clinical trials were performed long after a particular treatment outcome had been clearly demonstrated. Furthermore:

different endpoints (or outcomes) were used to determine successful treatment;
the endpoints were sometimes inappropriate; and
often too much emphasis was placed on statistical significance and ignoring the actual benefit of the treatment.

Top of page

About The Cochrane Library

The Cochrane Library is an electronic collection of databases published on the internet and also available on CD-Rom. It is updated quarterly in an effort to add to and keep the information current. The Library is made up of a number of parts.

The Cochrane Database of Systematic Reviews (CDSR) contains the published Cochrane reviews and protocols.

The Cochrane Central Register of Controlled Trials (CENTRAL) collates references to controlled trials in health care. These healthcare trial references are entered by Cochrane groups. The main way of finding health care studies is by looking in electronic databases (such as MEDLINE, EMBASE, CINAHL) using special search terms. Other ways are by asking experts in a particular health field and through hand searching journals.

The Database of Abstracts of Reviews of Effects (DARE) is a collection of structured abstracts and bibliographic references of systematic reviews of the effects of health care. It is developed by the Centre for Research and Dissemination, University of York, UK.

Methodological reviews and articles are also presented in The Cochrane Library.

In addition, each Cochrane group (termed an entity) has a section (module) in the Library that gives information on the group’s organisation, contact details, function, reviews, and other general information.

Accessing The Cochrane Library

Abstracts of reviews are readily accessible at www.cochrane.org/reviews. In countries such as Australia, Denmark, Finland, Ireland, Latin America, Norway and UK the full reviews are freely available as the governments of these countries have subscriptions to The Cochrane Library. Consumers who live in other countries and who wish to read a full review may need to access The Cochrane Library through a university, hospital or large public library.

A Cochrane Library Users Guide is available (http://www.nicsl.com.au/cochrane/index.asp) to help you find the information you want from The Cochrane Library.

Brief summaries (synopses) of Cochrane reviews are written for consumers and others to highlight the information in a review. A What’s New Digest summarises the newest reviews.

If you would like to make comments on any existing review in The Cochrane Library, you will find a special section for 'Comments and Criticisms' with the review.

Top of page

Systematic reviews

If someone decides to look critically at articles that have appeared in the medical or health literature on a particular topic they are said to be ‘reviewing the literature’. The authors may review, say, all the drug treatments available for one type of heart disease. A review is very clearly defined and sets out to find what evidence there is for prescribing one particular intervention or drug in a specific health condition, often in a certain group of people.

Examples of review topics are: Single dose celecoxib for acute postoperative pain; Artichoke leaf extract for treating hypercholesterolaemia; Chocolate avoidance for preventing migraine; Etidronate for treating and preventing postmenopausal osteoporosis.

What is a systematic review?

A systematic review summarises the results of available carefully designed healthcare studies (controlled trials) and provides a high level of evidence on the effectiveness of healthcare interventions.

The reviewers set about their task very methodically following, step by step, an advance plan that covers:

the way existing studies are found;
how the relevant studies are judged in terms of their usefulness in answering the review question;
how the results of the separate studies are brought together to give an overall measure of effectiveness (benefits and harms) – statistical techniques used to combine the results are called meta-analysis.

What is a protocol?

A protocol is the plan or set of steps to be followed in preparing a review. A protocol for a systematic review clearly describes why the review is needed (the review question), what the review is about (the healthcare context of the review), and how the reviewer authors will go about developing the review. It details how they will seek, select as relevant, critically appraise studies, and collect and analyse data (combine data and check for significance to the healthcare situation) from the included studies.

Cochrane protocols are published in the Cochrane Database of Systematic Reviews so that people can comment on them before the actual review has been carried out.

Top of page

How do I know a healthcare intervention works?

The aim of a systematic review is to thoroughly assess, by means of a set procedure, the best possible evidence about the effects of a healthcare intervention or treatment in a particular healthcare situation.

Healthcare studies are generally designed to assess the benefits, rather than the harms, of an intervention. Studies generally have a relatively short designated time period. Any possible harms of an intervention may be expected to occur less frequently and over a longer period of time than the studies cover.

About bias

The process of a review is clearly defined, before starting the actual review of the literature, to minimise associations of expectations of effects and other sources of bias. Bias is a systematic ‘error’ or mistake in the judgments and decisions made that influence the results of a study or a review. Bias differs from a ‘placebo effect’, which is where participants of a study (or assessors of the outcomes) perceive a beneficial effect, or harm, with an inactive treatment.

Synthesising evidence

The specific methods used in a review are carefully set out by The Cochrane Collaboration and are described in each review.

A Cochrane review is prepared and maintained using specific methodologies described in the Cochrane Reviewers’ Handbook.

Systematic reviews of randomised controlled trials provide the clearest evidence for the benefits of a healthcare intervention.

This is because the best way to assess the effects of a health care treatment is to use procedures that reduce the influence of chance effects and associations of cause and effect. Individual expectations on the part of a service provider, assessor and the person receiving an intervention can all contribute to modifying observed findings from a healthcare study. Randomised controlled trials where none of these people know the exact intervention a study participant is receiving (intervention under investigation, a placebo, or a comparator) may be expected to provide the best evidence.

Comparing groups can be misleading

By assessing the health of the two comparative groups in a study after their treatments, we can tell which intervention is more successful – but only if the two groups of people were very similar before treatment began. Otherwise we might be misled. For instance, one group may become healthier not because their treatment was better but because they were younger, not so ill, at less risk of ill health before treatment began, or even self selected to a particular intervention because of a particular personality trait, for example, people who chose to take a hormone may have wanted to stay younger and be more active.

Randomised controlled trials

Randomised controlled trials are studies that are rigorously designed. People are allocated to intervention groups in a way that minimises the chances of predicting which treatment group a study participant is in. The intervention under investigation is compared against a well-known intervention or an inactive treatment (placebo). Studies are controlled so that participants have similar associated care in all ways other than the intervention. Ideally, depending on the type of intervention, the service provider is unaware of which group a participant is in and those assessing outcomes are also unaware – this is termed ’blinding’.

The strength of evidence for a particular intervention can be increased further by systematically looking at (reviewing) all available randomized controlled trials that have been reported relevant to a particular healthcare situation.

It is important to search thoroughly for all studies

Many people are needed to properly test an intervention. This is more than can be recruited into a single trial; it is also important to investigate the intervention in different populations. Furthermore, the technical aspects of a particular randomised controlled trial may not have been implemented properly, for one reason or another. The effects of these shortcomings can be minimised by grouping results of a number of studies.

The results of randomised controlled trials may be published in any one of thousands of journals world wide. Indeed some studies are not published at all. In reality the studies found most easily tend to have over-optimistic results and finding reliable information about the effects of care is particularly difficult when there are negative results (the intervention is no better than placebo or another treatment). Sometimes published trials are too small to provide a conclusive result in their own right - as to whether a treatment really does work. Consequently, to find out about a healthcare intervention it is worth searching research literature thoroughly to see if the answer is already known. This may require considerable work over many months, but it will be much less work than conducting a new randomised controlled trial. This process also will not unnecessarily exclude people from effective interventions because of allocation to a placebo (or inactive treatment) group.

Discussions are underway in The Cochrane Collaboration as to how qualitative studies can be used to add to the information obtained from controlled studies - those that consider outcomes measured in numerical terms (and so are termed quantitative studies). Qualitative measures include ‘quality of life’ and lifestyle changes obtained from detailed questionnaires. Qualitative studies may also use narrative interviews where participants are asked to talk about their experiences around sets of semi-structured questions and prompts to explore particular issues that information is needed on for a study.

Top of page

What consumers can, and cannot, get from systematic reviews

Systematic reviews ask a very specific research question about a particular intervention in a clearly defined group of people who have a clear health condition or problem. Reviews provide powerful information on the state of knowledge about a healthcare intervention and whether that intervention is an effective treatment of a healthcare condition.

Reviews:

cannot offer a guideline for treatment, especially if a person differs from those defined in the review. Individuals may have accompanying health problems, be in a different healthcare setting, or receive more than one intervention, for example;
follow stringent guidelines as to what types of studies are included and how healthcare measures of effectiveness can be expressed and combined;
may consider outcomes other than the one you are interested in and may not look at long term effects of an intervention;
may only find studies that are limited in the healthcare setting in which they take place;
may provide conclusions that are limited because of the question asked and/or the studies that were found.

Reviews are dependent on the availability of studies and the information these studies sought or obtained.

Healthcare studies differ dramatically in what they look for and how well they are carried out and, therefore, how much weight one can put on their conclusions. Part of the reason for performing systematic reviews is to reduce the effects of these shortcomings. Issues of conflict of interest and corporate funding of healthcare studies are also important considerations in drawing conclusions from any study.

Reviews are better suited to assess benefits rather than harms.

Well-designed healthcare studies generally set out to determine the efficacy of a healthcare intervention. Information on potential harms is less well investigated.

Carefully controlled studies take place over a limited period of time so that the researchers can account for all people who entered the study from beginning to the end of the study. Harms are generally less common than benefits and may be apparent over a different time period. This may be, for example, only in the long term so that the intervention would have to be given to more people for a long time period for adverse effects to be studied effectively.

Participants of studies are selected to reduce the risk of other problems interfering with the efficacy of an intervention. How selective this process is needs to be carefully considered when assessing the relevance of a study to an individual.

Randomised controlled trials are expensive to run. They are very time consuming and multiple factors may limit how many participants are involved, the outcomes measured and the length of the trial. How many people complete the study is also very important.

Top of page

Levels of evidence for healthcare interventions

The National Health and Medical Research Council of Australia (1999) defines the ‘dimensions of evidence’ using three main areas.

1. Strength of the evidence

Level of evidence: the study design used – a systematic review of all relevant randomised controlled trials is the highest level, followed by at least one randomised controlled trial, then a pseudo-randomised trial
Quality of evidence: the methods used to minimise bias within a study design
Statistical precision: the degree of certainty about the existence of a true effect

2. Size of effect

How much the determined intervention effect is above a ‘no apparent effect’ value for clinically relevant effects

3. Relevance of the evidence

How appropriate the outcome measure is for the healthcare problem, and its usefulness in measuring effectiveness of treatment

Using a measure of the variability of results – confidence intervals

Adapted from AD.Oxman Checklists for review articles. BMJ 1994;309:648-51

Level I. For a randomised controlled trial, the lower limit of the confidence interval (expressed as a range) for a measure of effect is still above a meaningful benefit in healthcare terms

Level II. For a randomised controlled trial, the lower limit of the confidence interval (expressed as a range) for a measure of effect is less than a meaningful beneficial effect in healthcare terms; but the point estimate of effect still shows effectiveness of the intervention

Lower levels of evidence

Level III. Measures of effectiveness are taken from non-randomised studies of groups of people where a control group has run concurrently with the group receiving the intervention being assessed

Level IV. Measures of effectiveness are taken from non-randomised studies of groups of people where intervention effects are compared with previous or historical information

Level V. Evidence is from single case studies

Confidence interval (CI):

Even studies perfectly designed and carried out may show variable results because of the play of chance. CI covers the likely range of the true effect. For example, the result of a study may be that 40 per cent (95% CI 30% to 50%) of people are helped by a treatment. That means that we can be 95 per cent certain the true effect is between 30 and 50 per cent.
(Smart Health Choices How to make informed health decisions by Judy Irwig, Les Irwig and Melissa Sweet, Allen and Unwin 1999)

Top of page

Cochrane Groups

Cochrane Review Groups

Different groups exist for different health conditions: International cocllaborative review groups cover important areas of health care diseases and conditions. Review groups are responsible for producing and maintaining Cochrane reviews on specific health care questions. You will see in The Cochrane Library, for example, a Cochrane Consumers and Communication Group, Cochrane Epilepsy Group, Cochrane Heart Group and a Cochrane Pregnancy and Childbirth Group.

The activities of each group (or entity in Cochrane language) are monitored and co-ordinated by one person for each group, known as the review group co-ordinator (RGC). This person manages the day to day running of the group and is usually the contact person. The co-ordinating editor leads the group and is responsible for the quality and subject of reviews.

Each group attracts members with a variety of backgrounds, experience and expertise, who contribute to the process of developing systematic reviews. They may be doctors, nurses, researchers, health advisers, consumers and caregivers.

Cochrane Fields

Fields cover health care in a broader sense than do review groups. These may include a major section of health care such as cancer, the setting of care (e.g. primary care), the type of patient/consumer (e.g. older persons), the type of provider (e.g. nurses), or the type of intervention (e.g. vaccines). The role of fields is to facilitate the work of collaborative review groups and to ensure that Cochrane reviews appropriate to an area of interest are both relevant and accessible to service providers and consumers.

Each field works to:

identify relevant healthcare trials and make them accessible in a specialised register;
ensure the proper representation of its specialist area of health care in review groups;
act as a liaison point between the entities within The Cochrane Collaboration and the specialist area of health care;
promote the accessibility of Cochrane reviews.

The principal contact person in a field is its field co-ordinator.

Cochrane Centres

Cochrane centres provide a range of services designed to support collaborative review groups in their area and to facilitate the review process. They serve as a regional source of information about The Cochrane Collaboration, provide support to Cochrane contributors within a defined geographical area and promote access to The Cochrane Library. Each centre has a director.

The Cochrane Consumer Network (CCNet)

The Consumer Network supports consumer participation within The Cochrane Collaboration, internationally. The Network is available to any active consumer. Its mission is to enable and support consumers in contributing to the function of collaborative reviews groups and other Cochrane entities. The Network enables communication with other consumers, provides a sense of belonging within The Cochrane Collaboration, links and dissemination of information from Cochrane reviews.

Top of page