Redesigning the Cochrane highly sensitive search strategy for identifying RCTs in MEDLINE using statistical analysis to improve sensitivity and precision

Julie Glanville, Carol Lefebvre, Jeremy Miles, Janette Camosso-Stefinovic

Background: The original Cochrane Highly Sensitive Search Strategy (HSSS) for identifying RCTs in MEDLINE was designed by one of the authors (CL) in 1993 using standard subjective techniques, based on knowledge of the MEDLINE thesaurus (MeSH) and recommendations from experts in the field. The search strategy terms were not objectively derived and were not tested on an independent gold standard of known reports of RCTs. This strategy was first published in the BMJ in 1994 [1] and is included in the Cochrane Reviewers Handbook as Appendix 5b. Since this strategy was first designed, efforts to improve the identification of RCTs in MEDLINE have included the introduction of additional index terms in MeSH, improved indexing by US National Library of Medicine (NLM) indexers and the MEDLINE re-tagging project co-ordinated by the US Cochrane Center in co-operation with the NLM.
Objectives: To improve the current strategy by designing a highly sensitive and precise strategy, using objective statistical methods, to identify RCTs in MEDLINE.
Methods: We identified a gold standard set of known reports of RCTs and a comparator group of non-trials in MEDLINE. We conducted textual frequency analysis of the gold standard to identify frequently occurring terms. The presence of these terms in both the gold standard and the comparator sets was analysed using logistic regression in SPSS. The resulting discriminating terms were grouped into a range of strategies optimizing sensitivity or precision. The strategies were tested out in MEDLINE.
Results: New strategies with varying degrees of sensitivity and precision will be presented.
Conclusions: Using statistical analysis it has been possible to redesign the Cochrane HSSS and improve both its sensitivity and precision with respect to retrieving RCTs. It has also been possible to offer a precise and concise but sensitive strategy for the busy clinician.
References: 1. Dickersin K, Scherer R, Lefebvre C. Identifying relevant studies for systematic reviews. BMJ. 1994;309:1286-1291.

Ottawa 2004 O-054