Artificial intelligence (AI) tools are increasingly being used in evidence synthesis. But how do we decide whether using an AI tool is appropriate, and what are the considerations for performance and responsibility? This session, part of the series: Being responsible when using AI for evidence synthesis, will:-
- Highlight expectations of evidence synthesists and users of AI tools, and
- Introduce the framework on responsible handover of an AI tool to the evidence synthesis community (available in the Responsible use of AI in evidence SynthEsis (RAISE) recommendations and guidance; https://osf.io/fwaud/overview).
It will open by introducing the expectations of evidence synthesists using AI tools as detailed in RAISE and Cochrane’s joint position statement of AI use in evidence synthesis. This will be followed by highlighting the responsible handover framework and its five domains; what is the purpose of the tool; where have the training, testing and validation data come from; is the AI tool validated and performing sufficiently well for use; usability and user capability; and transparency, licenses, availability & documentation.
The session will end with discussion on how to interpret the findings after using the framework, as well as considerations for planning to use an AI tool in a systematic review and reporting its use.
It is open to everyone. Evidence synthesists and Cochrane authors, methodologists, AI tool developers, as well as anyone else interested in understanding the expectations for responsible AI use in evidence synthesis, such as funders, editors and publishers, trainers, or organisations or groups that produce evidence synthesis.
Presenter bios
Jo-Ana Chase is the Methods Implementation Editor in Cochrane’s Evidence Production and Methods Directorate. She works closely with cross-functional teams and methodologists to ensure rigorous, up-to-date evidence underpin Cochrane’s trusted reviews. Her work involves strategic methods support for Cochrane and its methods community, methods best practice implementation, including responsible use of AI, and continuous improvement in Cochrane. She is also a co-investigator on the Cochrane Evaluation of (Semi-) Automated Review methods (CESAR) project.
Sean Gardner is a Data Scientist in Cochrane's Central Executive Team (CET). His work supports use of Cochrane's rich data sources across the organisation, and involves both developing and evaluating AI-based evidence synthesis tools. He is interested in efforts to empower the Cochrane community to critically appraise applications of AI in their own domains.
