Chair (Stream 1): Youping Li (Vice Dean and Associate Professor of Medicine, The First University Hospital, Chengdu)
Chair (Stream 2): Bernie Towler (Australasian Cochrane Centre)
The effects of blinding articles for quality assessment
Authors: Arianne Verhagen, Henrica de Vet, Rob de Bie, Fons Kessels, Maarten Boers, Paul Knipschild
Background: A study of the literature is by definition non-experimental and therefore open to the same forms of bias as other observational studies. In order to prevent review bias some authors require blinded assessment of the quality of the articles. Systematic reviews performed in our department are blinded for authors, institutes where the study had been performed, the journal in which the paper was published and all information about the results of the intervention. This form of blinding is very time consuming. The aim of this study is to assess if our method of blinded reviewing is essential in preventing review bias.
Objectives: To assess if blinded reviewing differs from non-blinded reviewing.
Methods: We have used articles from a systematic review about the efficacy of balneotherapy in patients with arthritis. We excluded cross-over designs. In total 18 reviewers participated. They were university staff members and graduated students with epidemiological backgrounds. All reviewers followed training to ensure that the reviewers used the forms and procedures in identical ways. The computer designed a random table to guarantee that each article was scored 10 times (5 times blinded and 5 times non-blinded).
Results: A set of 12 experimental studies were used. All 18 reviewers completed the criteria lists and each of them scored on average 8 articles. The quality scores between the blinded and open assessment of the articles do not differ much. There is hardly any difference in ranking between the blinded and non-blinded assessment.
Conclusions: Although we reviewed only a small number of trials using reviewers with a high level of epidemiological knowledge, we found no evidence for review bias in quality assessment of clinical trials.
State-of-the-Art in Ophthalmology Randomized Controlled Trials
Authors: Roberta Scherer and Kay Dickersin; University of Maryland School of Medicine, USA
Background: The validity of systematic reviews depends on the quality of the randomized controlled trials (RCTs) included. Review of the state-of-the-art of RCTs in a discipline provides feedback to those in the area as to the overall acceptability of trials available for systematic reviews.
Objective: To assess the state-of-the-art of ophthalmology RCTS.
Methods: Eight ophthalmology journals were handsearched for RCTs published from 1988 to 1994. Reports that did not include patient data or only described study methods were excluded. RCT design characteristics were extracted using a pre-tested form. Multiple RCTs within a single report were analyzed separately and data from multiple reports emanating from a single RCT were combined on a single form.
Results: We found 527 RCTs; 97% (510/527) were randomized and 3% (17/527) used quasi-randomization. Inclusion criteria were reported in 97%, and exclusion criteria in 75% of RCTs. The method of randomization allocation concealment was reported in 213 RCTs (40%) and included: lists (12%), numbered bottles (10%), sealed envelopes (5%), telephone to central facility (3%), and other (10%). Stratification was reported in 13%, and blocking in 6% of trials. Masking was reported in 72% of RCTS: 51% masked patients; 47% persons delivering treatment; 68% persons measuring outcome; and 3% persons analyzing results. Reports included data from all randomized patients in analyses 55% (292/527) of the time. There were 235 reports that did not contain complete data on all patients, either because data were not available (99/235; 42%) or because data were excluded from the analysis (136/235; 58%).
Conclusions: Investigators of ophthalmology RCTs usually report masking, but infrequently report methodology related to randomization or use of intention-to-treat analyses when patients are lost to follow-up.
The nature and source of bias in Review Articles: The example of Chronic Fatigue Syndrome
Authors: John Joyce, Simon Wessely. Institute of Psychiatry, UK.
Background: Chronic Fatigue Syndrome is a controversial entity in that potentially relevant data to its study spans many disciplines. It therefore offers an excellent opportunity to test whether in an area requiring more than usually good reviewing skills an adequate level of methodological rigour has been present. The intense media interest surrounding this particular subject amplifies the importance of any revealed biases.
Objectives: To examine 1) the initial selection of literature on which each review is based 2) how conclusions on aetiology and treatment and citation of important papers of significant positive or negative findings is influenced by the authors personal and professional biases including discipline and nationality
Methods: All editorials, general reviews and reviews of treatment of CFS between 1980 and 1996 from both English and non English language journals were included. Any papers clearly labelled as dealing with a specialist aspect of CFS were excluded. We searched MEDLINE, EMBASE (BIDS), CURRENT CONTENTS and PSYCHLIT using keywords chronic fatigue, postviral (post viral) and myalgic encephalomyelitis. Methodological quality was assessed as follows
Results: The number of relevant articles identified was over 300 (306).
Conclusions: Preliminary results confirm our hypothesis that overviews of CFS have accessed relevant important literature in different disciplines pertinent to it in a less than systematic manner and have been marred by nonselective citation.
Survey of Refereeing Practices of Cochrane Review Groups
Authors: Lisa Bero, Veronica Yank, Drummond Rennie
Background: Pre-publication refereeing by peers is one method of assuring the quality of Cochrane Reviews.
Objectives: To determine the processes and tools CRGs use for the pre-publication peer refereeing of protocols and reviews and to facilitate the sharing of this information amongst the CRGs.
Methods: We sent a 26-question survey in e-mail to all CRG administrators or co-ordinating editors. The questions focused on the refereeing methods CRGs apply to different forms of developing reviews and how they are formalized and implemented.
Results: Seventeen of 19 CRGs responded. The majority of CRGs have peer refereeing policies for protocols (13/17) and for reviews (14/17), although only 7 of 17 CRGs have a written policy, circulated in a paper manual or guide (5/7), in a newsletter (6/7), or by other means (3/7). Responsibility for implementing the peer review policy lies solely with the administrator in 5/17, solely with the editor in 4/17, and is shared in 5/17. The majority of CRGs use referees internal to the CRG (14/17), external to the CRG (12/17), and external to the Cochrane Collaboration (12/17). Six maintain a database or index file of referees. Of the CRGs that have sent materials to referees, the methods used are hardcopy by post (1/14), e-mail attachment (9/14), disk in post (7/14), fax (7/14), and file transfer protocol (6/14). Few CRGs have policies for resolving disputes between referees and authors of a review (3/17), have mechanisms for dealing with post-publication criticism after it has been published (2/17), or are conducting research related to peer review (1/17). Ten of 17 CRGs requested assistance from the San Francisco Center in providing them with information and standards on refereeing processes, and in evaluating their effectiveness.
Conclusions: Our survey shows variation between CRGs in policies of pre-publication review by referees and in methods of implementing such review.
Towards a better understanding of what the "level of evidence" is and how this can be evaluated
Authors: Nadine Bossard, Jean-Pierre Boissel, Margaret Haugh, The French Cochrane Centre, France
Background: Actors in health systems make decisions that should be based on the best available evidence. This raises important issues of what is evidence, how can it be evaluated and graded. Numerous quality scales and checklists have been developed, some of which aim to quantify the level of evidence. However, several limitations restrict the use of these scales for the assessment of the level of evidence.
First, the level of evidence for an individual clinical trial is sometimes incorporated with the level of evidence for a systematic review of several trials.
Second, none of the existing scales include all three of the following components
Methods: Using the various published scales we have identified criteria which may be used for judging the level of evidence, without considering their respective weight. A panel of experts is being constituted and this panel will give their opinion on the level of evidence for a series of clinical trials. We will then model the weights accorded by the experts to our criteria, and using multi-criteria methods (commonly used in health economics research) we will define the hierarchical organisation. A parallel approach will examine the epistemological and sociological aspects, since the nature of the evidence depends on the hypothesis formulated, the means used to verify it, the interpretation of the results, and their generalisation. We will investigate if it is possible to use a scale to determine the level of evidence, since values vary under different circumstances and can change over time.
Scientific rigor and qualitative research
Authors: Carolyn Emden, University of South Australia, Australia. Heather Hancock, University of South Australia
Background: Scientific rigor is widely regarded as being exemplified by the Randomised Controlled Trial (RCT); indeed the RCT is termed the 'gold standard' of research within Cochrane literature. Matters are not so clear cut within the realm of qualitative research however (the term qualitative research being commonly used to denote 'any research which is not quasi experimental or experimental, through which numerical data are obtained' [Burns, 1989]). The intention of this paper is to encourage debate, and make some proposals, about what might reliably constitute scientific rigor in qualitative research. The paper contributes to a growing interest within and beyond Cochrane circles about qualitative research and the kind of evidence it can offer health care practitioners.
Approach: A report of literature across several disciplines provides the substantive content of the paper. As well, the topic is related to a research project currently being undertaken by the authors: a study of midwives' attitudes towards research before and after exposure to the Cochrane Pregnancy and Childbirth Database within their clinical setting.
Outcomes: The study of qualitative research in terms of its scientific defensibility is a relatively new field. There are multiple (potentially conflicting) frameworks available to guide the degree of importance one should place upon a single piece of qualitative research, and few guides to meta analyses. Available reviews are unreliable in the sense they are not systematic; for example, they do not follow any particular format. A proposal is put forward as to how experts in a particular discipline or field of practice can take account of, and make judgements about, this literature and reach some consensus about how they wish to systematically review qualitative research in their speciality, and further, how they might set about developing a mechanism for meta analyses of findings.
Meta-analysis of observational studies: Safety of home birth
Author: Ole Olsen
Background: Archie Cochrane awarded "the wooden spoon" to obstetrics, partly because "The specialty missed its first opportunity in the sixties to randomise the confinement of low risk pregnant women at home and in hospital".
Objective: To examine the safety of planned home birth backed up by a modern hospital system compared with planned hospital birth. Design: Meta-analysis of 6 controlled observational studies. Confounding was controlled either through restriction, matching or in the statistical analysis. Setting: The Western World. Subjects: 24092 selected and primarily low risk pregnant women. Main outcome measures: Perinatal mortality supplied with morbidity measures: Apgar score and maternal laceration. Intervention rates were noted.
Results: The perinatal mortality was not significantly different in the two groups (OR=0.87, 95% Cl 0.54-1.41). The principal difference in the outcome was a lower incidence of low Apgar scores (OR=0.55; 0.41-0.74) and a lower incidence of severe lacerations (OR=0.67; 0.54-0.83) in the home birth group. Medical interventions were much less common in the home birth group: induction (statistically significant ORs in the range (0.06-0.39), augmentation (0.26-0.69), episiotomy (0.02-0.39), operative vaginal birth (0.03-0.42) and caesarean section (0.05-0.31). No maternal deaths were observed in any of the studies.
Some of the differences may be partly due to bias. However, the findings regarding morbidity are supported by randomised clinical trials of elements of birth care relevant for home birth, and the finding relating to mortality is supported by large register studies comparing variously specialised hospital settings.
Conclusion: Home birth offers a safe and acceptable alternative to hospital confinement for selected pregnant women, and reduces the incidence of medical interventions.
The licorice allsorts approach: review and synthesis of a mixed bag of research
Author: Sophie Hill
Background: Randomised trials are seen as the gold standard in a panoply of research methods. However, much of the research which concerns itself with problems as defined by consumers of care - patients, carers, members of the public - is usually a very mixed bag in terms of methods used. What are the issues in a review and synthesis (or meta-analysis) of a mixed bag? How can we consume (and enjoy the benefits) of the liquorice allsorts, in addition to the gilded sweeties?
Objective: This paper will highlight some issues for discussion in the systematic review of research by many methods, ie, a range of quantitative and qualitative methods.
Method: Three reviews were conducted. Each involved a review and synthesis of research about consumers' concerns with, and experiences of care. The research included case studies, focus groups, interviews and questionnaires (all types), literature reviews and trials. There was a deliberate strategy to seek out research by all parties: consumer/community groups, academics (medical and social), and hospital and government departments.
Results: Each review was an input to a policy development process. The synthesis of issues for consumers was used alongside other data reviews in formulating policies to improve care.
Conclusions: There are several critical issues. Research results are complementary, not additive. Some types of research give us only glimpses - and murky at that if poorly done. But each work must be assessed according to the standards of each method, not according to a hierarchy. Different kinds of research give different insights, and so should not be discarded. In some fields, much of the experience of care is assumed to be unproblematic and so little if any research about people's concerns has been done. These gaps are made clear by this kind of review.
The efficacy of folic acid (FA) and folinic acid (FNA) reducing methotrexate (MTX) side effects in rheumatoid arthritis (RA)
Authors: Zulma Ortiz, Beverley Shea, Maria Suarez Almazor, David Moher, George Wells, Peter Tugwell
Background: MTX is an antimetabolite used in the treatment of RA and is considered to be the second-line drug of first choice. MTX's side effects occur in about 60% of patients. Some side effects mimic complicated folate deficiency. Several studies have been conducted to determine if folate supplementation has a beneficial effect reducing MTX side-effects, without altering its efficacy, but the answer remains uncertain.
Objective: To determine the efficacy of FA and FNA to reduce gastrointestinal and haematologic side effects of low-dose MTX in patients with RA, and whether or not folate supplementation alters MTX efficacy.
Methods: Using the Cochrane Musculoskeletal Group search strategy, we selected all double-blind, placebo-controlled randomised clinical trials in patients who fulfilled the American Rheumatism Association's criteria for RA, older than 18 years, treated with low dose MTX (<20 mg/week) concurrently with folate supplementation. The outcomes were the reduction of GI and/or haematologic side effects, and the efficacy of MTX in patients who received folate supplementation. Treatment effect was calculated using a fixed effect model (Peto). Sensitivity analysis was done to compare the results with different doses and different quality of trials.
Results: Six RCTs fulfilled the inclusion criteria. Haematologic side effects could not be analysed. A significant reduction in the odds of GI side effects of 68% was shown. Considering trials that use a dose of FA or FNA less than 10mg/week, there was a significant reduction in GI side effects of 75%.
Conclusions: Folate supplementation has a protective effect, reducing MTX related GI side effects, without affecting MTX efficacy. No evidence has shown that FA or FNA is the most effective.
Efficacy of 904 nm laser therapy in musculo-skeletal disorders: a systematic review
Authors: Rob de Bie, Arianne Verhagen, Ton Lenssen, Riekie de Vet, Frans van den Wildenberg
Background: Previous reviews show promising results of low dose laser therapy in the treatment of musculoskeletal disorders. However, no clear definition is given about dosage used. Also all possible wavelengths are assembled in one review. Recent studies show wave length and dose specific effects. It is therefore sensible to update previous reviews according to used wavelength.
Objective: To examine the effects of 904 nm laser therapy in musculoskeletal disorders.
Methods: All RCTs and CCTs on the efficacy of 904 nm laser therapy published until 1996 are collected. All language articles were eligible, provided an English abstract was presented. If necessary, translations of non-European language articles are provided for. Articles are reviewed independently by two reviewers using the 'Maastricht' criteria list, with additional criteria adapted from the preliminary results of the Delphi criteria list and the Jadad criteria list.
Results: 28 articles met the inclusion criteria. There was no apparent difference in quality between older and newer studies. Description of randomization procedure and drop-outs was poor, and sample sizes were small. Although very easy to accomplish, blinding of therapist and observer was rarely done.
Conclusions: 904 nm lasertherapy seems effective in establishing fast pain relief and quicker functional recovery, when compared to placebo laser therapy. More specifically, for rheumatoid arthritis, posttraumatic joint disorders, myofascial pain and trigeminal neuralgia, laser therapy seems to have therapeutic value.This conclusion is inevitably prudent as the methodological quality of the studies was rather poor.
A systematic review of the efficacy of common interventions for acute low back pain
Authors: Maurits van Tulder, Bart Koes, Lex Bouter
Background: There are many treatment options for acute low back pain, but little is known about the optimal treatment strategy.
Objective: To systematically evaluate the efficacy of the most common therapeutic interventions for acute low back pain.
Methods: A systematic computerized literature search in the MEDLINE, EMBASE and PsycLIT databases was conducted, and studies were included in the review if the interventions were randomly allocated, if the results (exclusively or separately) concerned patients with acute low back pain and if the article was published in English. A rating system was used to assess the strength of the evidence, based on the methodological quality of the randomized clinical trials, the relevance of the outcome measures and the consistency of the results.
Results: 81 Trials of 11 interventions were included in the review. The number of trials varied widely with regard to the interventions involved. In general, the methodological quality of the trials was poor; only 35% of the trials were considered to be of high quality. Various methodological flaws were identified. Strong evidence was found for the effectiveness of muscle relaxants and NSAIDs. The evidence in favor of other conservative types of treatment was only moderate or limited due to the low methodological quality of the trials involved. For some interventions no evidence at all was found due to the contradictory results.
Conclusions: There is much room for improvement of the quality of the design, execution and reporting of randomized clinical trials. A rating system for summarizing the available evidence on the basis of randomized clinical trials may be a useful tool in future systematic reviews. We suggest that the validity of the trials, the relevance of the outcome measures and the consistency of the results should be incorporated in such a rating system.
A systematic review of the efficacy of common interventions for chronic low back pain
Authors: Maurits van Tulder, Bart Koes, Lex Bouter
Background: A large variety of therapeutic interventions is available for the treatment of chronic low back pain, but the efficacy of most of these interventions has not yet been proved beyond doubt.
Objective: To systematically evaluate the efficacy of the most common therapeutic interventions for chronic low back pain.
Methods: A systematic computerized literature search in the MEDLINE, EMBASE and PsycLIT databases was conducted, and studies were included in the review if the interventions were randomly allocated, if the results (exclusively or separately) concerned patients with chronic low back pain and if the article was published in English. A rating system was used to assess the strength of the evidence, based on the methodological quality of the randomized clinical trials, the relevance of the outcome measures and the consistency of the results.
Results: 80 Trials of 14 interventions were included in the review. The number of trials varied widely with regard to the interventions involved. In general, the methodological quality of the trials was poor; only 25% of the trials were considered to be of high quality. The most prevalent methodological problems concerned the size of the study population, the adequacy of compliance, the avoidance of co-interventions, the blinded effect measurements, the adequacy of the randomization procedure, the blinding of patients, and the description of drop-outs. Strong evidence was found for the effectiveness of spinal manipulation, back schools and exercise therapy, especially for short-term effects. The evidence in favour of other conservative types of treatment was only moderate or limited, or there was no evidence at all.
Conclusions: The quality of the design, execution and reporting of randomized clinical trials should be improved in order to establish strong evidence for the efficacy of the various therapeutic interventions for chronic low back pain.
A systematic review of interventions for shoulder pain
Authors: Rachelle Buchbinder, Monash University, Australia. Rick Glazier, University of Toronto, Canada. Sally Green, Monash University, Australia.
Background: Shoulder pain is a common condition with significant morbidity. Many forms of therapy have been advocated, but evidence of their efficacy is not well documented.
Objectives: To perform a series of seven separate reviews of randomised controlled trials for shoulder pain, assessing:
Methods: The search strategy sought to identify all relevant trials.
Computer Based Searches: MEDLINE, EMBASE, CINAHL and Science Citation Index (SCISEARCH) searches back to 1966.
Hand Searches of major textbooks, bibliographies of relevant literature, the fugitive literature, and the subject indices of relevant journals back to 1966.
Each identified study was assessed for possible inclusion by two independent reviewers based on the blinded methods sections. The major determinant for inclusion was that the study be a randomised controlled trial and that the outcome assessment be blinded.
The same reviewers then assessed the methodological quality of the included studies based upon assessment of the blinded methods and the blinded results. This weighted the analysis. Data was extracted, facilitated by the Cochrane software, REVMAN.
Results: 58 trials have been identified and, while results are yet to be finalised, to date few demonstrate high methodological ranking. Final results will be available for presentation at the 4th International Cochrane Colloquium.
A systematic review of randomised trials on the efficacy of interventions for shoulder pain
Authors: Daniëlle van der Windt, Geert van der Heijden, Andrea de Winter, Bart Koes, Lex Bouter, Paul Knipschild
Background: Patients with shoulder disorders are treated with a variety of interventions, including physiotherapy, steroid injections and NSAIDs. However, the controversy on the efficacy of these interventions persists.
Objective: To systematically evaluate the efficacy of common interventions for shoulder disorders.
Methods: A systematic computerized literature search in Medline and Embase was conducted, supplemented with citation tracking. Studies were selected if interventions were randomly allocated to patients with shoulder disorders. Since validity of study outcomes may depend heavily on methodological quality, the methods were assessed systematically by two blinded independent reviewers. Next, differences between intervention groups in success-rate and corresponding 95% confidence intervals were calculated for each study.
Results: 21 trials on NSAIDs, 16 on steroid injections and 20 physiotherapeutical studies were included. The quality of methods was poor; only ±19 studies were assigned a score exceeding 50% of the maximum attainable score. The studies were often flawed by lack of blinding, inadequate study size, drop-outs and loss-to-follow-up higher than 20%. The method assessment was frequently hampered by incomplete information about randomisation, prognostic comparability, compliance and co-interventions. Few papers on physiotherapy reported point estimates and corresponding distribution measures for relevant outcome measures. Statistical pooling was further obstructed by considerable heterogeneity of study populations, interventions, and outcome measures.
Conclusions: The evidence in favour of the efficacy of interventions for shoulder disorders is scarce. This particularly holds for physiotherapy. Most studies that seem to be credible do not show significant differences compared with control treatment. Superior short-term efficacy of injections is demonstrated in two trials of relatively good quality. The evidence in favour of NSAIDs seems stronger, but it is unclear whether the use of NSAIDs is more favourable than analgesics, despite the higher risk of adverse reactions. For the Cochrane Meeting the strength of evidence will be graded using a framework derived from the method proposed by Guyatt et al (JAMA 1995;274:1800-4).
First and second year effects in calcium-bone density trials
Authors:Dorothy Mackerras (Menzies School of Health Research, Darwin), Thomas Lumley (NHMRC Clinical Trials Coordinating Centre, Sydney, Australia)
Background: The parameter reported in all studies assessing the effect of increasing calcium intake is the average annual rate of loss in bone density calculated over the duration of the trial. The underlying assumption is that the rate can be extrapolated beyond the observed duration of the trial. As calcium is a normal part of the diet, this approach includes the 'wash-in' period in the active treatment group which lasts for 6-12 months. Hence, this approach could lead to incorrect estimates of the longterm usefulness of calcium in reducing bone loss. We investigated whether the rate of bone loss differs in the first and second year of the trials.
Methods: Trials published in English using calcium supplements or dairy products as the treatment and change in bone density as the outcome were identified from reviews and a Medline search conducted in June 1994. Inclusion criteria were: a concurrent, randomised control group, intention-to-treat analysis and at least 2 years of follow-up in post-menopausal women. Combination therapy arms involving drugs with strong effects (e.g. oestrogen) were excluded. The baseline, 1 year and 2 year results were read by overlaying a transparent grid on the enlarged graphs. A weighted mean slope for each year was calculated and the difference between them tested under several covariance assumptions.
Results: Of 12 studies meeting the inclusion criteria, 6 included a graph or a table showing the effects by year. The percent of baseline bone density lost in the second year was greater than in the first year in the femoral neck: -1.28% (95% confidence interval: -0.61 to -1.95) and the spine: -0.88 (95% CI: -0.4 to -1.33). These differences are quite substantial in relation to the postulated benefit of calcium. There was no difference for the proximal radius: 0.03% (95% CI:-0.46 to 0.52). Differences in the other bones varied and had wide confidence intervals. These results were not sensitive to changing the covariance assumptions.
Conclusions: These results indicate that, at least for the femoral neck and spine, conducting a formal meta-analysis using currently published results is likely to over-estimate the longterm effectiveness of calcium supplementation in retarding bone loss.
The efficacy of balneotherapy in patients with arthritis; a systematic review
Authors: Arianne Verhagen, Henrica de Vet, Rob de Bie, Fons Kessels, Maarten Boers, Paul Knipschild
Background: Bathing in water has been frequently and enduringly used in classical medicine and is still a very popular form of treatment for arth-ritis in many European countries and in Israel. The term balneotherapy comes from the Latin balneum (bath). The aim of balneotherapy (or hydrotherapy) is to maintain or improve functional mobility. Some authors note that the development of rheumatology as a science began at the spas.
Objectives: A systematic review to assess the efficacy of balneotherapy in patients with arthritis.
Methods: Experimental studies, both randomized and non-randomized, published untill 1995 are collected. The language should be in English, French, Dutch or German. Two reviewers independently assess the methodological quality by using a criteria list.
Results: A total of 36 studies were found. On the basis of incomprehensible language 13 studies were excluded. Only 7 randomized and 7 non-randomized experimental studies met the inclusion criteria. The 'quality score' obtained varied between 9 and 49 point out of a maximum of 100 points. Beneficial effects of balneotherapy were reported in 10 studies. Improvement of most clinical variables is seen. None of the randomized studies showed an appropriate statistical analysis to support their conclusions.
Conclusion: A systematic review should summarize the best available knowledge on a certain topic. Unfortunately nearly half of the studies were excluded based on the language. All studies show methodological flaws. The non-randomized studies show relatively more flaws than the randomized. In conclusion, the evidence for beneficial effects of balneotherapy is poor.
This page was last updated on 18 November 1996
Comments for improvement or correction are welcome.
georg.koch@cochrane.de