Skip to main content

Systematic Reviews in the Health Sciences - RBHS: Getting Started

This guide will introduce you to the Systematic Review process.

What is a systematic review?

Healthcare decision makers in search of reliable information comparing health interventions increasingly turn to systematic reviews for the best summary of the evidence.

Systematic reviews identify, select, assess, and synthesize the findings of similar but separate studies and can help clarify what is known and not known about the potential benefits and harms of drugs, devices, and other healthcare services.

Systematic reviews can be helpful for clinicians who want to integrate research findings into their daily practices, for patients to make well-informed choices about their own care, and for professional medical societies and other organizations that develop clinical practice guidelines.

Institute of Medicine.(March 2011) .Report Briefs: Finding what works in healthcare standards for systematic reviews. 

Do You Really Want a Systematic Review?

A systematic review attempts to identify, appraise and synthesize all the empirical evidence that meets pre-specified eligibility criteria to answer a given research question. Researchers conducting systematic reviews use explicit methods aimed at minimizing bias, in order to produce more reliable findings that can be used to inform decision making. (See Section 1.2 in the Cochrane Handbook for Systematic Reviews of Interventions.)

 

Data Considerations

•Do I have a clearly defined clinical question with established inclusion and exclusion criteria?

•Do I have a team of at least three people assembled?

•Do I have time to go through as many search results as we might find?

•Do I have resources to get foreign language articles appropriately translated?

•Do I have the statistical resources to analyze and pool data?

 

If you answered “No” to any of the first four questions, a traditional Literature Review will be more appropriate to do.

If you answered “No” to the last question, a meta-analysis will not be an appropriate methodology for your review.

 

The following further outlines the difference between a "Systematic Review" and a "Literature Review."

 

If you think you do not need a systematic review but still need a Literature Review that is exhaustive, but not protocol-driven, librarians can still assist.

How a Systematic Review Improves Healthcare Decisions

How to Read a Systematic Review and Meta-analysis

What are Systematic Reviews used for?

Systematic reviews are used to assist group and individuals make decisions to improve peole's health. That includes

  • Recommendations and guidelines
  • Benefit design, coverage and policy decisions
  • Public Policy
  • Performance Measures
  • Research Agendas
  • Individual Patient are
  • Patient Decisions

What to consider when getting started

Time - average systematic reviews requires about 18 months of preparation

Team - you need to be working with: subject experts to help clarify issues related to the topic; librarians who can develop the comprehensive search strategies and identify the appriopriate databases to search; reviewers who can screen abstracts and read the full text; statistician who can assist with the appropriate analysis of the data; and project leader who will cordinate and write the final report

Written protocol - you need a written protocol that outlines the methodology, including the rationale for the systematic review, key questions broken into PICO components, inclusion/exclusion criteria, and literature search for both published and unpublished literature, data abstraction and data management, assessment of methodological quality of individual studies, data synthesis, and grading the evidence for each key question.

Literature searching - you need to first identify systematic reviews that may already address the key questions; then identify the appropriate databases and conduct a comprehensive and detailed literature search that can be documented and duplicated; 

Citation management - you need working knowledge of EndNote or other software package to help manage the citations from the literature search

Guidelines for reporting - you need to use the appropraite guideline for reporting your review for publication 

 

Meta-Analysis vs Systematic Review

DEFINITION 1:  Many systematic reviews contain meta-analyses. Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analyses can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review (see Chapter 9, Section 9.1.3). They also facilitate investigations of the consistency of evidence across studies, and the exploration of differences across studies.

 

Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

  

DEFINITION 2:  A systematic review is an overview of primary studies that used explicit and reproducible methods.  A meta-analysis is a mathematical synthesis of the results of two or more primary studies that addressed the same hypothesis in the same way.  Although meta-analysis can increase the precision of a result, it is important to ensure that the methods used for the review were valid and reliable.

 

Greenhalgh T.  How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses.) BMJ 1997; Sep 13,315: 672-5 PMID 9310574. 

Steps in the systematic review process

  1. Develop an answerable question 
  2. Check for recent systematic reviews  
  3. Agree on specific inclusion and exclusion criteria 
  4. Develop a system to organize data and notes
  5. Devise reproducible search methods 
  6. Launch and track exhaustive search 
  7. Organize search results 
  8. Reproduce search results 
  9. Abstract data into a standardized format
  10. Synthesize data using statistical methods (meta-analysis)  
  11. Write about what you found  

What you will need to conduct a review

Time Committment

The average systematic review requires 18 months of work. “…to find out about a healthcare intervention it is worth searching research literature thoroughly to see if the answer is already known. This may require considerable work over many months…” (Cochrane Collaboration)

 

The suggested timeline for a Cochrane review is: 

  • Preparation of protocol 1 – 2 months
  • Searches for published and unpublished studies 3-8 months
  • Pilot test of eligibility criteria 2-3 months
  • Inclusion assessments 3-8 months
  • Pilot test of ‘Risk of bias’ assessment 3 months
  • Validity assessments 3-10 months
  • Pilot test of data collection 3 months
  • Data collection 3-10 months
  • Data entry 3-10 months
  • Follow up of missing information 5-11 months
  • Analysis 8-10 months
  • Preparation of review report 1-11 months
  • Keeping the review up-to-date 12 months

Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane-handbook.org.

 

 

Citation Managment System

You will need a citation management system like EndNote to handle the large number of citations that you will need to deal with.  EndNote is a program that helps users quickly collect and organize references from online sources (or elsewhere), create a searchable personal database, find and cite these references while writing, and create bibliographies formatted in their style of choice. The Libraries are licensed to provide download access to EndNote  for all RBHS users. EndNote can be downloaded from here.

Stay Organized

The goal is to keep records in the most systematic way possible so that all of your work can be reproduced. That means you should keep detailed records of the exact search you used for each database and that all your searches should have an end date so that the results can be reproduced exactly every time.

Keep…

  •  detailed records of each search in addition to saving searches in your personal accounts which are available in each database.

  •  all your citations in a citation management program (like EndNote) so you can easily and quickly manipulate them

  •  a spreadsheet organized by article and sub-organized by preliminary inclusion and exclusion criteria to track why you included and excluded articles for more in-depth review

  •  detailed notes of in-depth reviews for each article organized by specific criteria

 

Types of study designs

Meta-Analysis


A way of combining data from many different research studies. A meta-analysis is a statistical process that combines the findings from individual studies.

 Example: 

Deriving consensus on the characteristics of advanced practice nursing: meta-summary of more than 2 decades of research. 
UI: 24589642

Systematic Review

A summary of the clinical literature. A systematic review is a critical assessment and evaluation of all research studies that address a particular clinical issue. The researchers use an organized method of locating, assembling, and evaluating a body of literature on a particular topic using a set of specific criteria. A systematic review typically includes a description of the findings of the collection of research studies. The systematic review may also include a quantitative pooling of data, called a meta-analysis.

 Example:  

A systematic review of the effectiveness of advanced practice nurses in long-term care. Donald F. Martin-Misener R. Carter N. Donald EE. Kaasalainen S. Wickson-Griffiths A. Lloyd M. Akhtar-Danesh N. DiCenso A. Journal of Advanced Nursing. 69(10):2148-61, 2013 Oct.[Journal Article. Research Support, Non-U.S. Gov't. Review] UI: 23527481

Randomized Controlled Trial

A controlled clinical trial that randomly (by chance) assigns participants to two or more groups. There are various methods to randomize study participants to their groups. 

Example: 

Assessment of proprioceptive exercises in the treatment of rotator cuff disorders in nursing professionals: a randomized controlled clinical trial. Martins LV. Marziale MH.Revista Brasileira de Fisioterapia. 16(6):502-9, 2012 Nov-Dec.[Comparative Study. Journal Article. Randomized Controlled Trial] UI: 23117648

Cohort Study (Prospective Observational Study)


A clinical research study in which people who presently have a certain condition or receive a particular treatment are followed over time and compared with another group of people who are not affected by the condition.

  

Example

Central venous catheter placement by advanced practice nurses demonstrates low procedural complication and infection rates--a report from 13 years of service*. Alexandrou E. Spencer TR. Frost SA. Mifflin N. Davidson PM. Hillman KM. Critical Care Medicine. 42(3):536-43, 2014 Mar. [Journal Article. Observational Study] UI: 24145843

 

Case-control Study


Case-control studies begin with the outcomes and do not follow people over time. Researchers choose people with a particular result (the cases) and interview the groups or check their records to ascertain what different experiences they had. They compare the odds of having an experience with the outcome to the odds of having an experience without the outcome. 

Example: 

Nonuse of bicycle helmets and risk of fatal head injury: a proportional mortality, case-control study. Persaud N. Coleman E. Zwolakowski D. Lauwers B. Cass D. CMAJ Canadian Medical Association Journal. 184(17):E921-3, 2012 Nov 20. [Journal Article. Research Support, Non-U.S. Gov't] UI: 23071369

Cross-sectional study


The observation of a defined population at a single point in time or time interval. Exposure and outcome are determined simultaneously. 

Example: 

Fasting might not be necessary before lipid screening: a nationally representative cross-sectional study. Steiner MJ. Skinner AC. Perrin EM. Pediatrics. 128(3):463-70, 2011 Sep.[Journal Article. Research Support, N.I.H., Extramural] UI: 21807697

 

 

Case Reports and Series

A report on a series of patients with an outcome of interest. No control group is involved.

Example: 

A case study of pediatric pneumonia with empyema. Waldrep VB. Sloand E. Journal of Pediatric Nursing. 28(2):167-70, 2013 Apr.
[Case Reports. Journal Article] UI: 22771429

 

 

 Ideas, Editorials, Opinions

Put forth by experts in the field. 

Example: 

Preparing for leadership. Newland J.Nurse Practitioner. 39(2):6, 2014 Feb 15.[Editorial] UI: 24407246

 Animal Research Studies

Studies conducted using animal subjects. 

Example: 

Perfusion defatting at subnormothermic temperatures in steatotic rat livers. Liu Q. Berendsen T. Izamis ML. Uygun B. Yarmush ML. Uygun K. Transplantation Proceedings. 45(9):3209-13, 2013 Nov. [Journal Article. Research Support, N.I.H., Extramural. Research Support, Non-U.S. Gov't] UI: 2418278

Test-tube Lab Research

"Test tube" experiments conducted in a controlled laboratory setting.