Evidence Synthesis in the Social Sciences

Extract Data

Once you have assembled your studies and assessed them for bias and quality, you can begin extracting data. The extraction process gleans information from the studies you have systematically included in your review.

During your protocol, you should develop an extraction template or set of variables to use on each included study, in accordance with PRISMA.

The extraction process should be done by at least two people on your systematic review team. These team members should work independently from one another.

Sample Data Extraction Form
By University of South Australia Libraries

Guidelines for Extraction

PRISMA 2020 Item 9
"Specify the methods used to collect data from reports, including how many reviewers collected data from each report, whether they worked independently, any processes for obtaining or confirming data from study investigators, and, if applicable, details of automation tools used in the process"
PRISMA 2020 Item 10
"List and define all other variables for which data were sought (such as participant and intervention characteristics, funding sources). Describe any assumptions made about any missing or unclear information"
Cochrane Handbook, Chapter 5
Chapter on Data Extraction

What Extraction Method Should I Use?

Tool	Benefits	Limitations
Systematic Review Software (aka Covidence)	Review elements are housed in single system Discrepancies are automatically highlighted for resolution Can calculate interrater reliability Better assurance of blinding during the extraction process Read PDFs of articles and extract data in side-by-side panel	Subscription-based to create more than 3 reviews Steeper learning curve to create and customize extraction forms
Spreadsheets (Excel, Google Sheets)	Excel/Google Suite free to Rutgers Affiliates Easy to learn and use (i.e., extractors will be able to begin quickly compared to using other software) Easy to customize extraction fields	Manually review, find, and resolve discrepancies Increase in potential bias if all extractors are using or have access to the same file (e.g., issues with blinding data extracted) Potential for more errors and less accuracy due to manual data entry and review
Cochrane Revman	Free Compatible with Covidence Capabilities to write the entire review using this software	Steeper learning curve to learn new software
Survey or Form Software (Google Forms, Qualtrics, etc.)	Qualtrics is free to rutgers affiliates Better assurance of blinding during the extraction process Extractors may be more familiar with using this interface compared to systematic review software (Covidence)	Free versions may have limited question or response options Need to ensure data is downloadable or able to be exported in a useable format
Electronic documents (Word, Google Docs)	Free options available Easy to learn and use (i.e., extractors will be able to begin quickly compared to using other software) Easy to customize extraction fields	Manually review, find, and resolve discrepancies Increase in potential bias if all extractors are using or have access to the same file (e.g., issues with blinding data extracted) Potential for more errors and less accuracy due to manual data entry and review

Adapted from University of North Carolina Libraries

What Should I Extract?

The variables you choose to extract data from will be determined by your research questions. In considering what to extract, Cochrane has provided a comprehensive table of data aspects to consider:

Cochrane Checklist of Items to Consider in Data Collection
Cochrane Handbook Chapter 5.3, Table 5.3.a

Extraction plans and data collected is very dependent on the research questions and type of review you are hoping to conduct. If you plan to synthesize or analyze your data, you might want to collect additional information like sample sizes, effect sizes, dependent variables, reliability measures, pre-test data, post-test data, follow-up data, and statistical tests used.