Cochrane has always aimed to provide the right evidence at the right time. As we move into a more collaborative and AI-enabled era, delivering this goal will increasingly rely on structured data.
Gert van Valkenhoef, Cochrane’s Head of Data, sets out why we are improving Cochrane’s approach to data, the difference this could make, and what this means for our authors.
The big picture – why this is important
Without standardized, open systems, evidence synthesis data cannot be efficiently shared and reused. Data are instead constantly being re‑extracted, re‑assessed, and re‑formatted – a costly duplication of effort.
The Evidence Synthesis Infrastructure Collaborative (ESIC) uses a historical example to illustrate this; the UK’s railway network in the 1800s. During this period, train tracks owned by different companies were often incompatible with each other. When these tracks met, goods and passengers had to laboriously move trains. The railway companies spent much of the 19th century battling over, agreeing and then slowly implementing standard train track gauges – wasting a great deal of time and money.
We face a similar challenge today. Real-time, machine-readable evidence that can be adapted, reused, and integrated has the potential to enable trustworthy, actionable insights, answer pressing health questions, and reduce duplication.
Cochrane is committed to supporting the principles developed by ESIC, which include striving for open, timely (living), and re-usable data as an output. Evidence synthesis is a crucial step on the pathway to influencing decision-making, but systematic review reports will rarely be read as stand-alone documents by a policymaker or citizen. Data, therefore, is not just a critical input, but also a critical output of evidence synthesis, enabling the findings to be evaluated, contextualized, and incorporated into guidelines, decision aids, and more.
A recent report from Clarivate and the Institute for Scientific Information (ISI) also highlights the value of structured and stable data for research to be discovered, compared and evaluated effectively. Given this, we must move towards capturing Cochrane review data in the most usable form, preparing for a future where study data is extracted once and used to answer many different questions.
The successes of the Cochrane Central Register of Controlled Trials (CENTRAL), the COVID-19 study register and COVID-NMA project, and the MetaPsy initiative, among others, show that creating structured, living, datasets not tied to any one specific question does pay off, both as a net effort-saver for review authors, and for evidence users who want to approach the evidence in different ways. We expect this to become a routine part of how we work, especially as improved infrastructure and AI capabilities make the approach increasingly feasible.
What we’ve done so far
Data sharing to improve access to evidence has long been at the heart of Cochrane’s approach. For example, Cochrane reviews transparently include the full meta-analysis data tables. While useful, this way of publishing the data inherently limits how it can be re-used by leaving out key details of the studies as well as certain transformations of the data. Moreover, managing the data this way makes it difficult to transfer from extraction tools such as Covidence into RevMan – Cochrane’s software for producing high-quality, standardized systematic reviews.
Study centric data was designed to tackle these challenges. With study centric data management, authors enter study results at the study level, rather than at the analysis level. Our implementation of this approach strikes a balance between ensuring minimal effort to extract data while capturing sufficient information to enable the same data to be analyzed in many ways. It also allows extracted data to easily be transferred from Covidence, or other data extraction platforms, into RevMan (“interoperability”).
The design was co-created with members of the community and technology partners, and an initial version was made available in RevMan in 2019. Since April 2023, study centric data has been our recommended approach for data management in intervention reviews. The uptake of study centric data has increased significantly since then, and we see many reviews using it successfully: over half of reviews that published using study centric data used it for all their analyses, and nearly all used it for over half of their analyses. The study centric approach is also at the heart of ongoing changes to RevMan that put the summary of findings tables front-and-center in the review authoring process and more clearly and effortlessly link the summary of findings to the underlying analyses and data.
Since its inception, Cochrane has recognized the importance of updating evidence syntheses, and it was a pioneer in this field. Cochrane contributors were also instrumental in developing the living systematic review approach, taking updating to the next level. But it would be fair to say that our processes and technologies are not ideal for publishing living reviews. That needs to change.
What’s next?
In the next year, we have three main aims. The first two relate to improving our data in intervention reviews. First, we want to make study centric data the standard approach. Second, we want to ensure that the data underlying network meta-analyses are published consistently as study centric data.
This creates new opportunities for users of the evidence to re-use Cochrane review data, potentially avoiding the need to re-extract underlying study data. By adopting study centric data for all Cochrane intervention reviews, we’re establishing a new baseline for data quality and re-usability on the Cochrane Library. As the users of our evidence start counting on Cochrane reviews including study centric data, we’ll start to see new ways of using the evidence emerge. And as Cochrane authors start to push the boundaries of what study centric data can do, we’ll find out where we need to improve.
We know study centric data is not perfect, and the RevMan knowledge base clearly sets out when it can and can’t be used. We will continue to improve the functionality offered by study centric data. That said, we believe that it is a net benefit for review authors already, with advantages including:
- helping with calculations like combining arms, changing reference arms, and log scale transformations
- having better provenance of data and more easily re-using data without risking transcription errors
- importing all study data in one go
Our third aim in the next year is to investigate how living reviews may publish their dataset more frequently. For example, communicating interim findings, even when they don’t justify a full review update. This is complicated and so we will be asking Cochrane’s community and other interest holders for their insights to help inform our work on this aim, before we identify a solution. Adopting study centric data and strengthening living evidence are our next steps towards an open data system. We hope to eventually see study centric data being used to produce broader living datasets that can be used to answer many questions and inform multiple reviews.
What does this mean for Cochrane authors?
We will expect new intervention reviews to use study centric data. In cases where study centric data can’t be used, the previous method of data entry can be used, and authors will be asked to justify this. We also expect the data underlying network meta-analyses to be reported using study centric data. Once tools are available to help authors migrate existing reviews to study centric data, this expectation will also apply to updates.
We understand this might be new for many authors, and we want to make it easy for you. We have updated guidance in the RevMan knowledge base. In addition, we already have training on study centric data.
We’d like to get input from authors to make sure the policies and guidance are useful, and all concerns are addressed. If you would like to provide feedback, please use this survey link. We’ll announce the final changes to policies and expectations soon, and release additional guidance.
For living reviews, we hope to share more information later this year.
Final thoughts
Cochrane is investing in structured, living, open, and interoperable data not as a technical exercise, but as a foundation for the evidence system the world now needs. A system where data can be shared, duplication is minimized, and trustworthy insights can be generated quickly, transparently, and equitably.
By working together to build the foundation, authors and contributors across the Cochrane community will help shape this emerging ecosystem.
Find out more
Training on study centric dataGuidance in the RevMan knowledge baseProvide feedback on upcoming guidance via our survey
