Skip to Main Content

Systematic Reviews in the Health Sciences

This guide will introduce you to the Systematic Review process.

Databases / Information Sources

Grey Literature

Definition:

There are many definitions of grey literature, but it is usually understood to mean literature that is not formally published in sources such as books or journal articles. It may be described as ephemeral, invisible, informal, underground, etc., that is, literature that may be unevaluated, not peer reviewed.

Grey Literature exists in many formats: preprints; preliminary progress and advanced reports; institutional, internal, technical, and statistical reports; research memoranda; state-of-the-art reports; market research reports; reports of commissions and study groups; as well as theses, conference proceedings, technical specifications and standards
translations (not distributed commercially), bibliographies, technical and commercial documentation, official documents (issued in limited numbers)
.

Alberani VDe Castro Pietrangeli PMazza AM.  The use of grey literature in health sciences: a preliminary survey.  Bull Med Libr Assoc. 1990 Oct;78(4):358-63. 

Grey Literature Sources

Contains citations and summaries of journal articles, books, and technical reports in the field of psychology and psychological aspects of related disciplines.

CIRRIE facilitates the sharing of information and expertise between the U.S. and world-wide partners to improve the conditions of people with disabilities, located in the School of Public Health and Health Professionsat the University of Buffalo since 1999.

Use the Advanced Search.

The Grey Literature Network Service was founded in 1992. The goal of GreyNet is to facilitate dialog, research, and communication between persons and organizations in the field of grey literature.

The National Technical Information Service serves as the largest central resource for government-funded scientific, technical, engineering, and business related information available today.

OAIster is a union catalog of millions of records representing open access resources from collections worldwide.

The REHABDATA database from the National Rehabilitation Information Center (NARIC). spans almost 50 years of disability and rehabilitation research. More than 200,000 abstracts are available through this database, including approximately 115,000 records originally indexed in the Center for International Rehabilitation Research Information and Exchange (CIRRIE) database.

Web of Science indexes leading scholarly journals, books, proceedings, reports, and other formats. Its major subject areas include sciences, social sciences, and arts and humanities.

Database of dissertations and theses from institutions around the world, including records for international dissertations beginning in 1637 and U.S. dissertations beginning in 1861 (the year the first doctoral dissertation was accepted at a U.S. institution). 

Electronic Theses and Dissertations (ETD) have replaced printed theses and dissertations for most Rutgers graduate programs and are available through RUcore. Beginning with October 2007 degree date submissions, dissertations and theses that represent a terminal degree from the Graduate School--New Brunswick are submitted in online form only. Other graduate schools, including the Graduate School--Newark, the Camden Graduate School, the Edward J. Bloustein School of Planning and Public Policy, the Graduate School of Applied and Professional Psychology, and the Graduate School of Education, are participating in the program as well. The Mason Gross School of the Arts has made electronic submission optional.

Develop a Search Strategy

It is recommended that a search strategy should be designed by an SR trained librarian or information specialist, or with input from a librarian or an information specialist who have experiences in searching for systematic reviews.

Major steps in developing a search strategy:

  • Identify the major concepts of the topic
    • Population or problem
    • Intervention or exposure or area of interest
    • Comparator if applicable
    • Outcomes
  • Harvest search terms
    • Identify Medical Subject Headings (MeSH Terms) in PubMed or Medline via Ovid or other platforms.
    • Check the related terms or the tree structure of the MeSH for additional related terms.
    • Perform a preliminary search in PubMed. In the most relevant records, check the title, abstract, and subject headings fields for the relevant keywords or text words.
    • Consult the subject reviewers for their input of related terms.
  • Combine the search terms using Boolean operators
    • Within a major concept, connect the terms using OR so that all of the terms will be present in the results.
      • Using “ ” around a phrase to search the exact phrase, retrieving more smaller and more specific results.
      • Using ( ) around a phrase to search the phrase loosely, resulting in larger and more sensitive results.
      • Using MeSH terms to search for articles that were indexed with the searched MeSH headings, retrieving the most relevant results.
    • Combi ne the search strings of different major concepts with AND so that both concepts will be present in the search results. Note: it depends on your research question to decide whether or not to use AND to connect different concepts’ search strings.
  • Test and validate the search strategy by performing test searches
    • Test the search strategy by copy the search string and paste it in the search box of a selected database, e.g. PubMed.
    • Ask the subject reviewers to examine the search results to check if the results are reasonably relevant. If there is little relevance, then the search strategy needs to be revised.
    • Following PRESS guidelines to peer review the search strategy.
    • Once the search strategy is validated, translate it to search other included databases and/or information sources.

Conduct a Literature Search

Once the databases and other information sources are selected and the search strategy has been translated for each source, you may start searching.

  • With a translated search strategy, perform a comprehensive search in each selected information source.
  • Consider searching grey literature sources e.g. ClinicalTrials.gov, ProQuest Dissertation & Theses Global, governmental/association websites, etc.
  • Document the search strategy for each information source for transparency and reproducibility.
  • Record the number of search results and screening outcomes for compiling the flowchart of study selection. If using Covidence, the flowchart will be automatically generated based on the numbers in the process.

Handsearching

What Is Handsearching?

Handsearching (also hand-searching and hand searching) is a manual method of scanning select journals from cover to cover, page-for-page for relevant articles in case they were missed during indexing. It is a methodical process of searching journal contents page by page (and, by hand) including articles, editorials, letters from readers, etc., to identify the relevant studies and complete the non-indexed searching in the databases. According to the Cochrane Handbook"...involves a manual page-by-page examination of the entire contents of a journal issue or conference proceedings to identify all eligible reports of trials.

"Handsearching may include checking the reference lists of journal articles, a technique called snowballing. In 2013, Craane et al found that "...hand search[ing] plays a valuable role in identifying randomised controlled trials" beyond Medline and Embase.

Craane B., Dikstra PU. (2012 Feb) Methodological quality of a systemtic review on physical therapy for temporomandibular disorders: influence of hand search and quality scales.  Clinical Oral Investigations 16(1) 295-303.

Who Does the Handsearching?

Handsearching is typically carried out by a trained handsearchers and must be documented along other search strategies. Recent research by health librarians suggests that hand-searching is still a requirement for the systematic review. Although keyword searching and reference harvesting reduce the need of doing handsearches, it is thought that hand-searching (due to non-existent, incomplete and/or inaccurate indexing) supplements the structured, documented searches in the biomedical databases.

There is a law of diminishing returns with searching online, and it may be necessary for searchers to consider hand-searching when it becomes clearer it will be more effective than searching online.

Handsearching in Systematic Reviews

Handsearching is a manual process of screening pre-defined and pre-selected peer-reviewed biomedical journals, conference proceedings and other publications for relevant materials that have been missed during the indexing process. Handsearching is widely considered necessary in the systematic review because it:

  • locates relevant articles poorly or inaccurately indexed or unindexed;
  • allows researchers to scan content quickly for relevant studies from the high-impact journals, and
  • ensures that relevant studies are not overlooked

Hand-searching increases the likelihood that no major relevant studies will be missed. Due to selective indexing in some databases and search tools and a tendency not to index supplements or special issues such as conference abstracts, handsearching is important for many if not most major research projects where comprehensive retrieval is required

Documenting Handsearching

Specific titles and date ranges searched for a systematic review should be included in the search strategies section. It should include journal titles, listed in alphabetical order, and the months and years that have been searched.

In addition, any websites that have been consulted, whether it be for the purposes of browsing for information, searching for grey literature or locating experts in the field, should also be documented.

Managing Search Results

When performing a comprehensive literature search, you should decide how to manage the search results.

  •  
  • Choose a citation management tool (e.g. EndNote, Zotero) to manage and organize search results.
  • Export references retrieved from each database into EndNote or Zotero. Note: You may also export the search results directly into an SR tool (e.g. Covidence, or Rayyan, JBI SUMARI (under EBP Tools), AHRQ’s SRDR+)
  • Remove duplicate references
  • Upload the remaining references to SR tools (e.g. Covidence, or Rayyan, or JBI SUMARI, SRDR+) for title and abstract screening