Skip to Main Content

Evidence Synthesis in the Social Sciences

Use of Artificial Intelligence in Systematic Review

Evidence synthesis projects are comprehensive and time consuming, to accelerate this process leveraging the use of artificial intelligence (AI) driven by human involvement. There have been various applications of AI while developing evidence synthesis projects: 

1. Screening and Eligibility

 AI tools can be employed to screen and assess the eligibility of articles based on predefined criteria. This helps in speeding up the initial phase of systematic reviews by automating the screening process. Refer to AI tools listed below developed for screening articles. 

2. Data Extraction and Automated Synthesis: 

Natural Language Processing (NLP) algorithms can be used to extract relevant data from articles, reducing the manual effort required for data extraction. This is especially useful in handling large volumes of information.

AI algorithms can support the synthesis of data by identifying patterns, relationships, and trends across studies. This can facilitate the extraction of meaningful insights from a large body of literature.

Note: It would be beneficial to consult or include a data expert in this step!

For More Information: 

Marshall, I.J., Wallace, B.C. Toward systematic review automation: a practical guide to using machine learning tools in research synthesis. Syst Rev 8, 163 (2019). https://doi.org/10.1186/s13643-019-1074-9

Van de Schoot, R., de Bruin, J., Schram, R. et al. An open source machine learning framework for efficient and transparent systematic reviews. Nat Mach Intell 3, 125–133 (2021). https://doi.org/10.1038/s42256-020-00287-7

 

Limitations of utilizing AI tools in Evidence Synthesis

Utilizing AI tools in evidence synthesis projects are relatively novel and requires caution while leveraging their use in your project. Following are some constrains of AI tools (adapted from Kahil et al., 2022): 

 

Limitations  AI Tools
Limited generalizability of results Reserach screener, SRA, Robotsearch, Robotreviewer, Abstrackr, R software with codes published publically, Exact, revtools in R package
 

Training AI tools before use

Swift review, In house data extraction tool, RCT tagger, Robotsearch, Abstrack, Swift-Review, ExaCT, Robotreviewer
Lack of integration  eSURFr, QuickClinical, Rayyan
Limited access of tools to public  Covidence, DistillerAl, Rayyan, Colandr, CADIMA, ReLis, PARSIFAL
 
Technical limitations LitSuggest, RobotReviewer,Google translate, NetMetaXL, Covidence, Abstrackr, DistillerSR, Robot Analyst, Raptor, Trials2rev
User's knowledge  SRA, Polygot Search, Robotreviewer
Limited resources (lack of funding) Evidence mapping tools, Abstrackr, RevmanHAL

AI resources for Evidence Synthesis

Covidence is a web-based software platform that streamlines the systematic review process. It assists with study screening, data extraction, and risk of bias assessment, making the systematic review process more efficient.

DistillerSR is web-based systematic review software that offers features such as study screening, data extraction, and collaborative analysis. It aids in managing the review process and has functionalities that can streamline evidence synthesis.

EPPI-Reviewer is a web-based systematic review software tool that supports various stages of the review process, including study selection, data extraction, and synthesis. 

Rayyan a free web application that assists with systematic reviews and allows for collaboration among multiple reviewers. 

IBM Watson has been utilized in healthcare research to help with evidence synthesis and analysis. 

Abstrackr helps in citation screening. 

For More Information: 

Alshami, A.; Elsayed, M.; Ali, E.; Eltoukhy, A.E.E.; Zayed, T. Harnessing the Power of ChatGPT for Automating Systematic Review Process: Methodology, Case Study, Limitations, and Future Directions. Systems 2023, 11, 351.

Mahuli, S., Rai, A., Mahuli, A. et al. Application ChatGPT in conducting systematic reviews and meta-analyses. Br Dent J 235, 90–92 (2023). https://doi.org/10.1038/s41415-023-6132-y

Blaizot A, Veettil SK, Saidoung P, Moreno-Garcia CF, Wiratunga N, Aceves-Martins M, Lai NM, Chaiyakunapruk N. Using artificial intelligence methods for systematic review in health sciences: A systematic review. Res Synth Methods. 2022 May;13(3):353-362. doi: 10.1002/jrsm.1553

Khalil H, Ameen D, Zarnegar A. Tools to support the automation of systematic reviews: a scoping review. J Clin Epidemiol. 2022 Apr;144:22-42. doi: 10.1016/j.jclinepi.2021.12.005.

(Kahil et. al., 2022)

RUTGERS.EDU | SEARCH RUTGERS.EDU

© , Rutgers, The State University of New Jersey

Rutgers is an equal access/equal opportunity institution. Individuals with disabilities are encouraged to direct suggestions, comments, or complaints concerning any accessibility issues with Rutgers websites to accessibility@rutgers.edu or complete the Report Accessibility Barrier / Provide Feedback form.