Systematic Reviews in the Health Sciences

Healthcare decision makers in search of reliable information comparing health interventions increasingly turn to systematic reviews for the best summary of the evidence.

Systematic reviews identify, select, assess, and synthesize the findings of similar but separate studies and can help clarify what is known and not known about the potential benefits and harms of drugs, devices, and other healthcare services.

Systematic reviews can be helpful for clinicians who want to integrate research findings into their daily practices, for patients to make well-informed choices about their own care, and for professional medical societies and other organizations that develop clinical practice guidelines.

Source: Institute of Medicine. (March 2011) .Report Briefs: Finding what works in healthcare standards for systematic reviews. 

--

What are systematic reviews used for?

Systematic reviews are used to assist group and individuals make decisions to improve people’s health. That includes

• Recommendations and guidelines
• Benefit design, coverage and policy decisions
• Public Policy
• Performance Measures
• Research Agendas
• Individual Patient are
• Patient Decisions

A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making. (See Section 1.2 in the Cochrane Handbook for Systematic Reviews of Interventions.)

Data Considerations

• Do I have a clearly defined clinical question with established inclusion and exclusion criteria?
• Do I have a team of at least three people assembled?
• Do I have time to go through as many search results as we might find?
• Do I have resources to get foreign language articles appropriately translated?
• Do I have the statistical resources to analyze and pool data?

If you answered “No” to any of the first four questions, a traditional Literature Review will be more appropriate to do.

If you answered “No” to the last question, a meta-analysis will not be an appropriate methodology for your review.

The following further outlines the difference between a "Systematic Review" and a "Literature Review."

If you think you do not need a systematic review but still need a Literature Review that is exhaustive, but not protocol-driven, librarians can still assist.

Murad MH, Montori VM, Ioannidis JP, Jaeschke R, Devereaux PJ, Prasad K, Neumann I, Carrasco-Labra A, Agoritsas T, Hatala R, Meade MO, Wyer P, Cook DJ, Guyatt G. How to read a systematic review and meta-analysis and apply the results to patient care: users' guides to the medical literature. JAMA. 2014 Jul;312(2):171-9. doi: 10.1001/jama.2014.5559. PMID: 25005654. https://pubmed.ncbi.nlm.nih.gov/25005654/ 

Time - average systematic reviews requires about 18 months of preparation

Team - you need to be working with: subject experts to help clarify issues related to the topic; librarians who can develop the comprehensive search strategies and identify the appropriate databases to search; reviewers who can screen abstracts and read the full text; statistician who can assist with the appropriate analysis of the data; and project leader who will coordinate and write the final report

Written protocol - you need a written protocol that outlines the methodology, including the rationale for the systematic review, key questions broken into PICO components, inclusion/exclusion criteria, and literature search for both published and unpublished literature, data abstraction and data management, assessment of methodological quality of individual studies, data synthesis, and grading the evidence for each key question.

Literature searching - you need to first identify systematic reviews that may already address the key questions; then identify the appropriate databases and conduct a comprehensive and detailed literature search that can be documented and duplicated; 

Citation management - you need working knowledge of EndNote or other software package to help manage the citations from the literature search.

Guidelines for reporting - you need to use the appropriate guideline for reporting your review for publication. 

DEFINITION 1:  Many systematic reviews contain meta-analyses. Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review (see Chapter 9, Section 9.1.3). They also facilitate investigations of the consistency of evidence across studies, and the exploration of differences across studies. 

Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

DEFINITION 2:  A systematic review is an overview of primary studies that used explicit and reproducible methods.  A meta-analysis is a mathematical synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way.  Although meta-analysis can increase the precision of a result, it is important to ensure that the methods used for the review were valid and reliable. 

Greenhalgh T.  How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses.) BMJ 1997; Sep 13,315: 672-5 PMID 9310574. 

1. Develop an answerable question 
2. Check for recent systematic reviews  
3. Agree on specific inclusion and exclusion criteria 
4. Develop a system to organize data and notes
5. Devise reproducible search methods 
6. Launch and track exhaustive search 
7. Organize search results 
8. Reproduce search results 
9. Abstract data into a standardized format
   
10. Synthesize data using statistical methods (meta-analysis)  
11. Write about what you found  

Time Commitment

The average systematic review requires 18 months of work. “…to find out about a healthcare intervention it is worth searching research literature thoroughly to see if the answer is already known. This may require considerable work over many months…” (Cochrane Collaboration)

 The suggested timeline for a Cochrane review is: 

• Preparation of protocol: ...............................................       1-2 months

• Searches for published and unpublished studies: ........       3-8 months
• Pilot test of eligibility criteria: .......................................       2-3 months
• Inclusion assessments:    ................................................      3-8 months
• Pilot test of ‘Risk of bias’ assessment:       ......................      3 months
• Validity assessments:       ................................................      3-10 months
• Pilot test of data collection: ..........................................        3 months
• Data collection: .............................................................        3-10 months
• Data entry: ....................................................................        3-10 months
• Follow up of missing information: ................................         5-11 months
• Analysis: ........................................................................        8-10 months
• Preparation of review report:       ...................................       1-11 months
• Keeping the review up-to-date:   ...................................        12 months

Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

Citation Management System

You will need a citation management system like EndNote to handle the large number of citations that you will need to deal with.  EndNote is a program that helps users quickly collect and organize references from online sources (or elsewhere), create a searchable personal database, find and cite these references while writing, and create bibliographies formatted in their style of choice. The Libraries provide download access to EndNote for all Rutgers students, staff and faculty free of charge. users. EndNote can be downloaded from here.  http://www.libraries.rutgers.edu/EndNote

Stay Organized

The goal is to keep records in the most systematic way possible so that all of your work can be reproduced. That means you should keep detailed records of the exact search you used for each database and that all your searches should have an end date so that the results can be reproduced exactly every time.

Keep…

•  detailed records of each search in addition to saving searches in your personal accounts which are available in each database.
   
•  all your citations in a citation management program (like EndNote) so you can easily and quickly manipulate them
   
•  a spreadsheet organized by article and sub-organized by preliminary inclusion and exclusion criteria to track why you included and excluded articles for more in-depth review
   
•  detailed notes of in-depth reviews for each article organized by specific criteria

Meta-Analysis

A way of combining data from many different research studies. A meta-analysis is a statistical process that combines the findings from individual studies.

 Example: 

Systematic Review

A summary of the clinical literature. A systematic review is a critical assessment and evaluation of all research studies that address a particular clinical issue. The researchers use an organized method of locating, assembling, and evaluating a body of literature on a particular topic using a set of specific criteria. A systematic review typically includes a description of the findings of the collection of research studies. The systematic review may also include a quantitative pooling of data, called a meta-analysis.

 Example:  

Randomized Controlled Trial

A controlled clinical trial that randomly (by chance) assigns participants to two or more groups. There are various methods to randomize study participants to their groups. 

Example: 

Cohort Study (Prospective Observational Study)

A clinical research study in which people who presently have a certain condition or receive a particular treatment are followed over time and compared with another group of people who are not affected by the condition.

Example: 

Case-control Study

Case-control studies begin with the outcomes and do not follow people over time. Researchers choose people with a particular result (the cases) and interview the groups or check their records to ascertain what different experiences they had. They compare the odds of having an experience with the outcome to the odds of having an experience without the outcome. 

Example: 

Cross-sectional study

The observation of a defined population at a single point in time or time interval. Exposure and outcome are determined simultaneously. 

Example: 

Case Reports and Series

A report on a series of patients with an outcome of interest. No control group is involved.

Example: 

 Ideas, Editorials, Opinions

Put forth by experts in the field. 

Example: 

 Animal Research Studies

Studies conducted using animal subjects. 

Example: 

Test-tube Lab Research

"Test tube" experiments conducted in a controlled laboratory setting.