Systematic Reviews in the Health Sciences

This guide provides an overview of systematic review, the major steps to conducting a systematic review, resources and tools for systematic reviews, a page on scoping reviews, and the Health Sciences Libraries (HSLs)' consultation and collaboration services in systematic reviews (SR) and scoping reviews (ScR). 

A systematic review, as a type of evidence syntheses, follows a rigorous and structured method with a well-defined, transparent, and replicable process to identify, evaluate, and synthesize the findings of all relevant individual studies on a particular topic or research question.

Key Characteristics:

1. Predefined Protocol: A systematic review is guided by a pre-specified protocol, which outlines the research question, inclusion/exclusion criteria, search strategy, data extraction methods, and analysis plan.
2. Comprehensive Search: It involves a comprehensive and exhaustive search of multiple databases and sources to identify all relevant studies, including unpublished literature (grey literature) to minimize publication bias.
3. Critical Appraisal: Each included study is critically appraised for methodological quality using standardized tools and criteria.
4. Reproducibility: The process is transparent and reproducible, allowing other researchers to follow the same steps and verify the findings.
5. Quantitative Synthesis: A systematic review often includes a meta-analysis, a statistical technique that combines the results of individual studies to provide a pooled estimate of the effect size.

What Are Systematic Reviews Used for?

Systematic reviews can be helpful for clinicians who want to integrate research findings into their daily practices, for patients to make well-informed choices about their own care, and for professional medical societies and other organizations that develop clinical practice guidelines. They are useful for:

• Recommendations and guidelines
• Benefit design, coverage and policy decisions
• Public Policy
• Performance Measures
• Research Agendas
• Individual Patient are
• Patient Decisions

DEFINITION 1:  Many systematic reviews contain meta-analyses. Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review (see Chapter 9, Section 9.1.3). They also facilitate investigations of the consistency of evidence across studies, and the exploration of differences across studies. 

Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

DEFINITION 2:  A systematic review is an overview of primary studies that used explicit and reproducible methods.  A meta-analysis is a mathematical synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way.  Although meta-analysis can increase the precision of a result, it is important to ensure that the methods used for the review were valid and reliable. 

Greenhalgh T.  How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses.) BMJ 1997; Sep 13,315: 672-5 PMID 9310574. 

How to Critically Appraise a Systematic Review?

How to critically appraise a systematic review - Part I < https://youtu.be/NSUk5FLbJoY>

How to critically appraise a systematic review - Part II <https://youtu.be/Ly__U-n4fiQ>

Two short videos by Dr. Shaneyfelt who is an Associate Professor in the Department of Medicine at UAB. He is a general internist who developed a passion for EBM after participating in the McMaster EBM course in 1999. His research interests have focused around clinical practice guidelines and the evaluation & teaching of EBM. He teaches EBM principles to students, residents and faculty. He is a past member of the Society of General Internal Medicine EBM Task Force. He is an associate editor of BMJ EBM journal.

A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making. (See Section 1.2 in the Cochrane Handbook for Systematic Reviews of Interventions.)

Do you really want to do a systematic review? Answer the following questions to determine:

• Do I have a clearly defined clinical question with established inclusion and exclusion criteria?
• Do I have a team of at least three people assembled?
• Do I have time to go through as many search results as we might find?
• Do I have resources to get foreign language articles appropriately translated?
• Do I have the statistical resources to analyze and pool data?

If you answered “No” to any of the first four questions, a traditional Literature Review will be more appropriate to do.

If you answered “No” to the last question, a meta-analysis will not be an appropriate methodology for your review.

The following further outlines the difference between a "Systematic Review" and a "Literature Review."

If you think you do not need a systematic review but still need a Literature Review that is exhaustive, but not protocol-driven, librarians can still assist.

The following table outlines the differences between a "Systematic Review" and a "Literature Review."

If you think you do not need a systematic review but still need an exhaustive but not protocol-driven literature review, librarians can still assist.

Time - you need a timeline for the project. The time required to conduct a systematic review can vary significantly based on several factors, including the scope of the review, the complexity of the topic, the size of the team, and the availability of resources. According to a study analyzing data from the PROSPERO registry, the mean project length from start to publication is about 67.3 weeks, or approximately 1.3 years​.* On average, conducting a systematic review can take anywhere from 6 months to 2 years. Here's a breakdown of the stages and approximate timeframes:

• Planning and Protocol Development: 1-3 months
• Literature Search: 1-2 months
• Screening and Selection of Studies: 2-6 months
• Data Extraction and Management: 1-3 months
• Quality Assessment: 1-3 months
• Data Synthesis and Analysis: 2-4 months
• Writing and Reporting: 2-4 months
• Peer Review and Revisions: 1-3 months

These timeframes can overlap, and the total duration will depend on the efficiency of the processes and the specific requirements of the review.

*Borah R, Brown AW, Capers PL, Kaiser KA. Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry. BMJ Open. 2017;7(2):e012545. Epub 20170227. doi: 10.1136/bmjopen-2016-012545. PubMed PMID: 28242767; PMCID: PMC5337708.

Team - you need to be working with: 

• subject experts to help clarify issues related to the topic
• librarians who can develop the comprehensive search strategies and identify the appropriate databases to search
• reviewers who can screen titles/abstracts and evaluate the full text
• statistician who can assist with the appropriate analysis of the data
• project leader who will coordinate and write the final report

Written protocol - you need a written protocol deposited in PROSPERO that outlines the methodology including:

• the rationale for the systematic review
• key questions broken into PICO/PECO components
• inclusion/exclusion criteria
• literature search for both published and unpublished literature
• screening search results
• data abstraction
• assessment of the risk of bias of individual studies
• data synthesis

Literature searching - you need to:

• perform preliminary searches for existing systematic reviews that may have already addressed the key questions
• harvest search terms
• develop search strategies
• identify the appropriate databases and other information sources
• translate search strategies for other databases
• conduct a comprehensive and detailed literature search that can be documented and duplicated

Citation management - you need working knowledge of citation management tools (e.g. EndNote, Zotero) to help manage the references from the literature search. You also need to use tools that are designed for screening references and/or performing other functions in conducting systematic reviews (e.g. Covidence, JBI SUMARI, Rayyan, etc.). For more information on how to use the tools, click here.

Guidelines for reporting - you need to use the appropriate guidelines for conducting and reporting your SRs.

• Cochrane Handbook for Systematic Reviews of Interventions - The Cochrane Handbook for Systematic Reviews of Interventions is the official guide that describes in detail the process of preparing and maintaining Cochrane systematic reviews on the effects of healthcare interventions.
• Methodological Expectations of Cochrane Intervention Reviews (MECIR)  - The Methodological Expectations of Cochrane Intervention Reviews (known as the MECIR Standards) are methodological standards to which all Cochrane Protocols, Reviews, and Updates are expected to adhere.
• PRISMA - PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. The current version is PRISMA 2020.
• PRISMA Flow Diagram - The flow diagram depicts the flow of information through the different phases of a systematic review. It maps out the number of records identified, included and excluded, and the reasons for exclusions.
• JBI Manual for Evidence Synthesis – This manual guides authors who wish to conduct systematic and scoping reviews following JBI methodologies. Each chapter is devoted to the synthesis of different types of evidence to address different types of clinical and policy-related questions. (Aromataris E, Lockwood C, Porritt K, Pilla B, Jordan Z, editors. JBI Manual for Evidence Synthesis. JBI; 2024. Available from: https://synthesismanual.jbi.global.)