Editor in Chief of the Cochrane Library, David Tovey, describes the journey of updating the Cochrane Library, which spans being a journal and a knowledge resource, and includes current and archive content, multi-lingual text, and a varied collection of databases.
It all began in April 2012 at Cochrane’s business meeting in Paris: a ‘strategic session’ aimed at agreeing a content strategy for Cochrane and our primary output, the Cochrane Library. My team had prepared well: there was broad approval for moves towards greater diversity in content, and more flexibility in its presentation and delivery to end users (readers). If that moment represents conception, it has been an awfully long gestation.
Readers less familiar with Cochrane may be uncertain about what constitutes the Cochrane Library. Is it a journal? Is it a knowledge database? Therein lies some of the complexity. The Library is a collection of knowledge databases. One of these, the best known and most used, is the Cochrane Database of Systematic Reviews (CDSR). This contains some 7600 systematic reviews, published over the past 25 years, many of which have been updated several times. The CDSR is also a hybrid journal containing some open access content and some available via subscription, which is largely organised on a national, regional or institutional basis. It has monthly issues, an impact factor, and is heavily accessed and cited. The Cochrane Central Register of Controlled Trials (CENTRAL) is a concentrated collection of (predominantly) randomised controlled trials, curated from a variety of sources. It is an essential part of the search for all high quality systematic reviews, whether produced inside Cochrane or not.
The recommendations that emerged from the Paris meeting formed the basis for goals 1 and 2 of the subsequent Strategy to 2020: to produce high quality, relevant reviews that inform health care decision making and to ensure that those reviews are accessed and used. The work to deliver on these goals can be tracked forward to our current Transformation Programme, which has brought together our 52 Cochrane Review Groups into 8 thematic Networks, and the development of our Content and Knowledge Translation strategies. These all reflect an awareness that our content needs to meet increasingly diverse and sophisticated decision maker needs and reflect emerging methods and data sources. We also need to find different ways to communicate review findings, based on a range of different user types (‘personas’ in tech development-speak), and including more diversity in language, presentation (or ‘packaging’) and delivery.
Fast forward from Paris to the café at the Novotel, Hyderabad in September 2014. Around a table in the late afternoon during the Cochrane Colloquium, Todd Toler and Deborah Pentesco-Murphy on behalf of our publishers, John Wiley and Sons laid out their proposals. In essence, after many frustrations, we all recognised that the Wiley Online Library platform, which had hosted the Cochrane Library for many years, was not able to accommodate Cochrane’s plans, without undermining its ability to host all Wiley’s other journals. We agreed that the solution was for Cochrane to be hosted separately, at Wiley’s expense, on a new platform, which would be identified through an RFP (Request for Proposals) process. The discussion energised us all, and it was not difficult to get clearance from our Governing Board to proceed. We had a plan.
A few months later, the venue is a Business School residential centre near London. The largest interview team I have ever experienced – seemingly about 20 people in the room, with more on the phone. Two of us from Cochrane. Four candidate teams over two days. Two clear leaders, and an eventual winner: a software company called Semantico, based in Brighton.
The development was meant to take 12 months. Six or seven months in, we were already behind schedule, when in a blink of an eye Semantico ceased to exist; acquired by Highwire a platform technology company based in Stanford USA with encouraging experience of having hosted major international journals. It may seem churlish now to point out that one of the influences in our choice of development partner was the personal commitment from Semantico’s founder and owner to stay the course. As it turned out not only did Semantico cease to exist but so did its entire Senior Management Team – submerged below the surface without so much as a ripple. The Semantico project manager, then and now viewed widely on all sides as one of the many unstinting heroes of the project, provides the only visible continuity, apart from the developers.
As time went on it became increasingly clear that our deadlines were not being met. The tri-partite development approach originally agreed by Cochrane, Wiley and Highwire was severely tested – we always knew that it was the greatest risk to the project. We got further and further behind schedule despite everyone’s best efforts, which were considerable. Our Governing Board looked over our shoulder, expressed its concern, but permitted us to proceed.
Most of us have had to familiarise ourselves with a new language and new processes. This is a world where ‘waterfall’ is an alternative to ‘agile’. Everyone seemingly agrees that ‘agile’ is the preferred method – leading to show and tell sessions we can attend following every two week ‘sprint’ where we can see the progress that is being made and provide feedback. There are ‘scrum teams’, ‘stand ups’ (brief functional meetings where comedy is not usually a feature), ‘tickets’ and some new acronyms: UX (user experience), UAT (user assessment testing), SMEs (subject matter experts). We talk of ‘SOS’ when we mean ‘Scrum of Scrums’, we document almost every breath in ‘Confluence’, and every task in ‘Jira’. None of the Cochrane editorial team has used these before but by the end of the project several of the team members could choose either as their specialist subject on a quiz show. We talk about ‘Epics’, by which we don’t mean ‘Gone with the Wind’, ‘stories’ (ditto) and ‘features’ (still ditto). You can complete a feature or story without signing off an epic. Or perhaps it is the other way around. Somewhere back in the distant past lies our comfort zone.
After all these tribulations, we have finally reached the first summit. Cochrane has a brand new Cochrane Library. It looks superficially similar to its predecessor so as not to alarm our frequent users, but with some important enhancements. Our multi-lingual content is easier to find and better presented. This is important. In the past three years we have seen a tripling of the number of people accessing the summaries on Cochrane.org, our own website. This has been almost entirely driven by the increases in use in Francophone and Spanish speaking countries. It is true that if you provide content that is easy to find and in people’s own languages, they will come. The new Cochrane Library incorporates more links between related pieces of content, internally and between the Library and external sites. This includes more links between the bibliographic records in CENTRAL and the reviews in CDSR, and more links from reviews to relevant guidelines. The Cochrane Library now includes content aimed at practising health professionals – Cochrane Clinical Answers - and a federated search of the Epistemonikos platform to identify non-Cochrane systematic reviews. There are other features and functionality available now, including some aimed at ensuring that the Cochrane Library meets the needs of, and links with other developments in the scientific publishing landscape such as Crossref, ORCID, Altmetric, Read Cube and others. Most importantly, there is the promise of many more to follow, including the potential to search by PICO parameters (Population, Intervention, Comparators, Outcomes), and the ability to publish different content types, such as methodological articles, rapid reviews and more varied forms of systematic review, and to package them in different ways customised around the needs of evidence users.
There is no denying that the journey has been harder than we all imagined in 2012. Perhaps that is the underlying message. There is a reason why project management language is now dominated by its technology influences. Nothing is simple when you are building a product that spans being a journal and a knowledge resource, includes current and archive content, multi-lingual text, and a varied collection of databases. If you are planning such a large technology project, my advice is to take 10 deep breaths and ask yourself: do you know everything you need to know? Are you sufficiently clear on your requirements and have you communicated them effectively? What will the implementation effort be, and how much will this distract you from everything else you want to achieve? Study it as hard and in as much detail as you can, and then consider whether to add a nought on to every calculation.
Nonetheless, across the three organisations, committed, skilled groups of people have worked tirelessly to deliver the new Cochrane Library. I am extraordinarily proud of the contributions all of our teams have made, whilst recognising the costs, the sacrifices and pain that the project has incurred. Throughout a long and arduous process, the relationships within and across our organisations have remained positive, cordial and collegiate. We are all pleased with what we have delivered and excited about the next steps. Please visit the Cochrane Library, and send me your feedback. We hope that it will become the ‘home of evidence’ that our ‘Strategy to 2020’ envisaged.
Editor in Chief