Evidence Synthesis in the Social Sciences

Grey literature refers to a diverse array of information that is generated outside of conventional publishing and distribution methods, often lacking proper representation in indexing databases (i.e. search engines). 

Grey literature can include unpublished surveys, government reports and publications, conference proceedings and abstracts, policy briefs and white papers, theses and dissertations etc. View an extensive list of Document Type in Grey Literature.

The approach to including and searching for grey literature will vary significantly depending on the nature of your research question, necessitating a tailored strategy that aligns with the specific characteristics associated with your area of investigation.

Finding grey literature involves exploring various non-conventional sources and repositories. Here are some effective ways to locate grey literature:

Specialized Databases

• Grey Literature Report is produced by the New York Academy of Medicine, this resources entails various public health and health disparity related material that can be utilized in social science reviews. 
• CADTH's Grey Matters resource is a free online tool for finding health-related grey literature.
• Grey Literature Network Service was produced to facilitate dialog, research, and communication between persons and organizations in the field of grey literature. They also hold internationals conferences for grey literature. 
• OpenGrey includes a comprehensive collection of grey literature produced in Europe. 
• ROAR: Registry of Open Access Repositories 
• OpenDOAR a directory of academic open access repositories, OpenDOAR is a project that lists and categorizes academic open access research repositories. 
• Ranking Web of Repositories identifies repositories worldwide. 
• Canadian Association of Research Libraries provides a list of repositories in Canada, including federal, provin- cial, and regional repositories.
• Digital Commons Network allows users to search across over 500 repositories using the Digital Commons institutional repository platform. 

Search Engines

Conduct comprehensive searches using search engines like Google Scholar, Google, and other scholarly databases. Include specific keywords and phrases related to your research topic to uncover relevant grey literature.

Tips to find grey literature in Google Scholar (adapted from Haddaway et al., 2015): 

1. Substantially more grey literature is found using title searches in Google Scholar than full text searches.

2. It is suggested that the greatest volume of grey literature in searches occurs at around 35 pages for title searches in Google Scholar. It is found that majority of grey literature begins to appear after approximately 20 to 30 pages of results. 

Steps for Structured Grey Literature Google Searching

• Look for Information from International Sources

Going to Google Advanced Search and limiting search results from a specific region is one of the easiest ways to access a varying range of grey literature hits. 

• Limit to File Type

  Limiting the file type to Adobe Acrobat PDF or MS Word may be especially good for structuring a search to only retrieve hits with more detailed content, such as evidence briefs or white papers. 

(Banato, 2018)

• Limit by Domain

Limiting by search domain is another means to exclude search hits from irrelevant sources, such as commercial sources

(Banato, 2018)

Government and NGO Websites 

Check government websites, policy institutes, and non-governmental organization (NGO) platforms for reports, policy papers, and research studies related to your topic of interest. Here are two database platforms that are supported by google search system. 

• NGO Search Database
• International Government Organization Search Database

Institutional Repositories

Explore the repositories of academic institutions, universities, and research organizations, which often house theses, dissertations, technical reports, and other unpublished research materials.

ProQuest has developed a Dissertations Data Repository that can be helpful in identifying grey literature from dissertations and theses. 

Conference Proceedings

Look for conference proceedings and abstracts from academic conferences, as these can often contain valuable research findings and studies that have not yet been formally published.

Where to find conference proceedings: 

• British Library Inside Conferences
• Conference Proceedings- Web of Science Index
• F1000 Research
• Scopus
• Web of Conferences

Professional Associations and Societies

Visit the websites of professional associations and societies within your field, as they may publish reports, white papers, and other forms of grey literature.

Preprint Servers

Preprint servers where researchers may share preliminary findings, unpublished manuscripts, and other forms of grey literature. Following are some helpful preprint servers in social sciences:

• Open Science Framework
• SOCOPEN
• SAGE publication preprints

Knowing when to stop searching for grey literature can be a nuanced decision and may depend on several factors. Here are some considerations to help you determine when it might be time to conclude your search:

• Relevance of Sources: Assess the relevance of the sources you've found. If the information aligns well with your research goals and contributes meaningfully to your work, it may be a sign that you've covered the necessary ground.
• Coverage of Perspectives: Consider whether your search has covered a diverse range of perspectives and sources. If you have obtained a comprehensive understanding of the topic, additional searching may yield diminishing returns.
• Search Exhaustion: If you find that you are repeatedly encountering the same sources or not discovering new and valuable information, it may be an indication that you've exhausted the available grey literature on your topic.
• Consultation with Peers or Experts: Seek feedback from peers, mentors, or subject matter experts. If others in your field believe you have covered the relevant literature, it may be time to conclude your search.
• Diminishing Returns: If additional searches consistently yield information of diminishing relevance or significance, it may be a sign that further efforts are not productive.

Read Kastner et al., 2007 to read more on "stop searching" strategy in systematic reviews. 

Advantages of including grey literature

• Broader Perspective
• Grey literature can provide a more comprehensive and holistic view of the research landscape, incorporating diverse perspectives and insights that may not be available through traditional published sources.
• Current and Timely Information
• Grey literature often includes up-to-date information, offering the latest research findings, policy developments, and other relevant data that may not have been formally published yet.
• Policy and Practice Relevance
• Grey literature frequently includes policy briefs, reports, and white papers, providing practical implications and applications that can be directly relevant to policy-making, practice, and decision-making processes.
• Diverse Data Types
• Incorporating grey literature enables you to utilize various data types, such as technical reports, conference proceedings, and working papers, which can contribute diverse perspectives and methodologies to your research.
• Filling Knowledge Gaps
• Grey literature can help fill knowledge gaps and address research questions that may not have been adequately covered by published academic literature, offering a more nuanced understanding of the research topic.
• Reducing Publication Bias
• Including grey literature can reduce the impact of publication bias by incorporating findings and insights that may not have met the stringent publication criteria of traditional peer-reviewed journals.

Limitations of Grey Literature 

• Quality Control Challenges
• Grey literature may lack the rigorous peer-review process associated with traditional academic publishing, leading to potential concerns about the quality and reliability of the information. Some forms of grey literature may be challenging to access, as they are often not widely disseminated or indexed, making it difficult to verify their authenticity and credibility.
• Variability in Reporting Standards
• Grey literature sources may exhibit a wide range of reporting standards and methodologies, making it crucial to critically evaluate and assess the credibility of the information presented.
• Risk of Outdated Information
• Some grey literature, especially in rapidly evolving fields, may contain outdated or superseded information that could impact the accuracy and relevance of your literature review.
• Lack of Citations and Attribution
• Grey literature often lacks proper citations and attributions, which can make it challenging to verify the original source and trace the information back to its primary author or organization.
• Considering these limitations, it is essential to exercise caution when integrating grey literature into your review, ensuring that you critically evaluate and validate the credibility and relevance of the sources to maintain the integrity of your research.
• Review our resources for Critical Appraisal to evaluate your literature. 

Searching the Grey Literature: a handbook for searching reports, working papers, and other unpublished research

Cochrane Grey Literature Databases