Use of Artificial Intelligence in Systematic Review
Evidence synthesis projects are comprehensive and time consuming, to accelerate this process leveraging the use of artificial intelligence (AI) driven by human involvement. There have been various applications of AI while developing evidence synthesis projects:
1. Screening and Eligibility
AI tools can be employed to screen and assess the eligibility of articles based on predefined criteria. This helps in speeding up the initial phase of systematic reviews by automating the screening process. Refer to AI tools listed below developed for screening articles.
2. Data Extraction and Automated Synthesis:
Natural Language Processing (NLP) algorithms can be used to extract relevant data from articles, reducing the manual effort required for data extraction. This is especially useful in handling large volumes of information.
AI algorithms can support the synthesis of data by identifying patterns, relationships, and trends across studies. This can facilitate the extraction of meaningful insights from a large body of literature.
Note: It would be beneficial to consult or include a data expert in this step!
For More Information:
Marshall, I.J., Wallace, B.C. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev 8, 163 (2019). https://doi.org/10.1186/s13643-019-1074-9
Van de Schoot, R., de Bruin, J., Schram, R. et al. An open source machine learning framework for efficient and transparent systematic reviews. Nat Mach Intell 3, 125–133 (2021). https://doi.org/10.1038/s42256-020-00287-7
Limitations of utilizing AI tools in Evidence Synthesis
Utilizing AI tools in evidence synthesis projects are relatively novel and requires caution while leveraging their use in your project. Following are some constrains of AI tools (adapted from Kahil et al., 2022):
| Limitations | AI Tools |
| Limited generalizability of results | Reserach screener, SRA, Robotsearch, Robotreviewer, Abstrackr, R software with codes published publically, Exact, revtools in R package |
|
Training AI tools before use |
Swift review, In house data extraction tool, RCT tagger, Robotsearch, Abstrack, Swift-Review, ExaCT, Robotreviewer |
| Lack of integration | eSURFr, QuickClinical, Rayyan |
| Limited access of tools to public | Covidence, DistillerAl, Rayyan, Colandr, CADIMA, ReLis, PARSIFAL |
| Technical limitations | LitSuggest, RobotReviewer,Google translate, NetMetaXL, Covidence, Abstrackr, DistillerSR, Robot Analyst, Raptor, Trials2rev |
| User's knowledge | SRA, Polygot Search, Robotreviewer |
| Limited resources (lack of funding) | Evidence mapping tools, Abstrackr, RevmanHAL |
AI resources for Evidence Synthesis
Covidence is a web-based software platform that streamlines the systematic review process. It assists with study screening, data extraction, and risk of bias assessment, making the systematic review process more efficient.
DistillerSR is web-based systematic review software that offers features such as study screening, data extraction, and collaborative analysis. It aids in managing the review process and has functionalities that can streamline evidence synthesis.
EPPI-Reviewer is a web-based systematic review software tool that supports various stages of the review process, including study selection, data extraction, and synthesis.
Rayyan a free web application that assists with systematic reviews and allows for collaboration among multiple reviewers.
IBM Watson has been utilized in healthcare research to help with evidence synthesis and analysis.
Abstrackr helps in citation screening.
For More Information:
Alshami, A.; Elsayed, M.; Ali, E.; Eltoukhy, A.E.E.; Zayed, T. Harnessing the Power of ChatGPT for Automating Systematic Review Process: Methodology, Case Study, Limitations, and Future Directions. Systems 2023, 11, 351.
Mahuli, S., Rai, A., Mahuli, A. et al. Application ChatGPT in conducting systematic reviews and meta-analyses. Br Dent J 235, 90–92 (2023). https://doi.org/10.1038/s41415-023-6132-y
Blaizot A, Veettil SK, Saidoung P, Moreno-Garcia CF, Wiratunga N, Aceves-Martins M, Lai NM, Chaiyakunapruk N. Using artificial intelligence methods for systematic review in health sciences: A systematic review. Res Synth Methods. 2022 May;13(3):353-362. doi: 10.1002/jrsm.1553
Khalil H, Ameen D, Zarnegar A. Tools to support the automation of systematic reviews: a scoping review. J Clin Epidemiol. 2022 Apr;144:22-42. doi: 10.1016/j.jclinepi.2021.12.005.
(Kahil et. al., 2022)