Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2023 Dec 1;12(1):226.
doi: 10.1186/s13643-023-02322-1.

Data extraction and comparison for complex systematic reviews: a step-by-step guideline and an implementation example using open-source software

Mohamed Afifi  1   2 , Henrik Stryhn  3 , Javier Sanchez  3
Affiliations
Meta-Analysis

Data extraction and comparison for complex systematic reviews: a step-by-step guideline and an implementation example using open-source software

Mohamed Afifi et al. Syst Rev. .

Abstract

Background: Data extraction (DE) is a challenging step in systematic reviews (SRs). Complex SRs can involve multiple interventions and/or outcomes and encompass multiple research questions. Attempts have been made to clarify DE aspects focusing on the subsequent meta-analysis; there are, however, no guidelines for DE in complex SRs. Comparing datasets extracted independently by pairs of reviewers to detect discrepancies is also cumbersome, especially when the number of extracted variables and/or studies is colossal. This work aims to provide a set of practical steps to help SR teams design and build DE tools and compare extracted data for complex SRs.

Methods: We provided a 10-step guideline, from determining data items and structure to data comparison, to help identify discrepancies and solve data disagreements between reviewers. The steps were organised into three phases: planning and building the database and data manipulation. Each step was described and illustrated with examples, and relevant references were provided for further guidance. A demonstration example was presented to illustrate the application of Epi Info and R in the database building and data manipulation phases. The proposed guideline was also summarised and compared with previous DE guidelines.

Results: The steps of this guideline are described generally without focusing on a particular software application or meta-analysis technique. We emphasised determining the organisational data structure and highlighted its role in the subsequent steps of database building. In addition to the minimal programming skills needed, creating relational databases and data validation features of Epi info can be utilised to build DE tools for complex SRs. However, two R libraries are needed to facilitate data comparison and solve discrepancies.

Conclusions: We hope adopting this guideline can help review teams construct DE tools that suit their complex review projects. Although Epi Info depends on proprietary software for data storage, it can still be a potential alternative to other commercial DE software for completing complex reviews.

Keywords: Complex; Data extraction; Database; Epi Info; Guideline; R; Systematic review.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Flowchart of the DE steps, Epi Info was applied in Steps 5 and 6 of the database building phase, and R libraries were applied in the data manipulation phase (Steps 9 and 10)
Fig. 2
Fig. 2
The tree diagram illustrates the hierarchical organisation of the data, including 5 entities arranged from top to bottom as STUDY, GROUP, OUTCOME, ARM and CONTRAST, along with their corresponding data items
Fig. 3
Fig. 3
The full ER diagram shows the relationships among the different entities in the database. Each box symbolised a single entity corresponding to an Epi Info form, except the ARM and CONTRAST, which were constructed as grids (i.e. table-like data entry fields) and added to the main form. The data items were listed within each entity. The lines with ‘1’ and ‘M’ markings show the 1:M relationships among the entities. The primary and foreign keys are indicated as GlobalRecordId and FKEY in Epi Info, respectively
Fig. 4
Fig. 4
The output of the compareDF library. The colour schemes facilitate the recognition of the discrepancies and agreements between the two reviewers. A single cell is coloured if it has changed across the two datasets. The discrepancies in the values in the first and second reviewer datasets were coloured green and red, respectively. Cells that did not change across the two datasets are coloured blue
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Jap J, Saldanha IJ, Smith BT, Lau J, Schmid CH, Li T, et al. Features and functioning of Data Abstraction Assistant, a software application for data abstraction during systematic reviews. Res Synth Methods. 2019;10(1):2–14. doi: 10.1002/jrsm.1326. - DOI - PMC - PubMed
    1. Chandler J, Churchill R, Higgins JPT, Lasserson T, Tovey D. Methodological expectations of Cochrane intervention reviews. Version 1.05, January 2018. 2019. Available at http://community.cochrane.org/mecir-manual. Accessed 18 Nov 2019.
    1. Li T, Vedula SS, Hadar N, Parkin C, Lau J, Dickersin K. Innovations in data collection, management, and archiving for systematic reviews. Ann Intern Med. 2015;162(4):287–294. doi: 10.7326/M14-1603. - DOI - PubMed
    1. Li T, Higgins JP, Deeks JJ. Chapter 5: Collecting data. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane handbook for systematic reviews of interventions 63 (updated February 2022). London: Cochrane; 2022. p. 109–41.
    1. Aromataris E, Munn Z. JBI manual for evidence synthesis. 2020. Available from https://synthesismanual.jbi.global. Accessed July 2020.

Publication types