Reporting of methods to prepare, pilot and perform data extraction in systematic reviews: analysis of a sample of 152 Cochrane and non-Cochrane reviews
- PMID: 34742231
- PMCID: PMC8571672
- DOI: 10.1186/s12874-021-01438-z
Reporting of methods to prepare, pilot and perform data extraction in systematic reviews: analysis of a sample of 152 Cochrane and non-Cochrane reviews
Abstract
Background: Previous research on data extraction methods in systematic reviews has focused on single aspects of the process. We aimed to provide a deeper insight into these methods by analysing a current sample of reviews.
Methods: We included systematic reviews of health interventions in humans published in English. We analysed 75 Cochrane reviews from May and June 2020 and a random sample of non-Cochrane reviews published in the same period and retrieved from Medline. We linked reviews with protocols and study registrations. We collected information on preparing, piloting, and performing data extraction and on use of software to assist review conduct (automation tools). Data were extracted by one author, with 20% extracted in duplicate. Data were analysed descriptively.
Results: Of the 152 included reviews, 77 reported use of a standardized extraction form (51%); 42 provided information on the type of form used (28%); 24 on piloting (16%); 58 on what data was collected (38%); 133 on the extraction method (88%); 107 on resolving disagreements (70%); 103 on methods to obtain additional data or information (68%); 52 on procedures to avoid data errors (34%); and 47 on methods to deal with multiple study reports (31%). Items were more frequently reported in Cochrane than non-Cochrane reviews. The data extraction form used was published in 10 reviews (7%). Use of software was rarely reported except for statistical analysis software and use of RevMan and GRADEpro GDT in Cochrane reviews. Covidence was the most frequent automation tool used: 18 reviews used it for study selection (12%) and 9 for data extraction (6%).
Conclusions: Reporting of data extraction methods in systematic reviews is limited, especially in non-Cochrane reviews. This includes core items of data extraction such as methods used to manage disagreements. Few reviews currently use software to assist data extraction and review conduct. Our results can serve as a baseline to assess the uptake of such tools in future analyses.
Keywords: Data extraction; Evidence synthesis; Systematic review methods.
© 2021. The Author(s).
Conflict of interest statement
The authors declare that they have no competing interests.
Figures
Similar articles
-
Folic acid supplementation and malaria susceptibility and severity among people taking antifolate antimalarial drugs in endemic areas.Cochrane Database Syst Rev. 2022 Feb 1;2(2022):CD014217. doi: 10.1002/14651858.CD014217. Cochrane Database Syst Rev. 2022. PMID: 36321557 Free PMC article.
-
The future of Cochrane Neonatal.Early Hum Dev. 2020 Nov;150:105191. doi: 10.1016/j.earlhumdev.2020.105191. Epub 2020 Sep 12. Early Hum Dev. 2020. PMID: 33036834
-
Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions.Cochrane Database Syst Rev. 2014 Oct 1;2014(10):MR000035. doi: 10.1002/14651858.MR000035.pub2. Cochrane Database Syst Rev. 2014. PMID: 25271098 Free PMC article.
-
Development, testing and use of data extraction forms in systematic reviews: a review of methodological guidance.BMC Med Res Methodol. 2020 Oct 19;20(1):259. doi: 10.1186/s12874-020-01143-3. BMC Med Res Methodol. 2020. PMID: 33076832 Free PMC article. Review.
-
Vaccines for the common cold.Cochrane Database Syst Rev. 2022 Dec 14;12(12):CD002190. doi: 10.1002/14651858.CD002190.pub6. Cochrane Database Syst Rev. 2022. PMID: 36515550 Free PMC article. Review.
Cited by
-
Data extraction and comparison for complex systematic reviews: a step-by-step guideline and an implementation example using open-source software.Syst Rev. 2023 Dec 1;12(1):226. doi: 10.1186/s13643-023-02322-1. Syst Rev. 2023. PMID: 38041161 Free PMC article.
-
Efficacy of automated insulin delivery (AID) systems in type 1 diabetes: protocol of a systematic review and network meta-analysis of outpatient randomised controlled trials.BMJ Open. 2023 Oct 10;13(10):e074317. doi: 10.1136/bmjopen-2023-074317. BMJ Open. 2023. PMID: 37816564 Free PMC article.
-
Carotid revascularisation versus medical treatment for asymptomatic carotid artery stenosis.Cochrane Database Syst Rev. 2024 Jul 5;7(7):CD015499. doi: 10.1002/14651858.CD015499. Cochrane Database Syst Rev. 2024. PMID: 38967132 Free PMC article.
-
Measurement properties of the Mental Health Literacy Scale (MHLS) validation studies: a systematic review protocol.BMJ Open. 2024 Apr 2;14(4):e081394. doi: 10.1136/bmjopen-2023-081394. BMJ Open. 2024. PMID: 38569702 Free PMC article.
-
The normality assumption on between-study random effects was questionable in a considerable number of Cochrane meta-analyses.BMC Med. 2023 Mar 29;21(1):112. doi: 10.1186/s12916-023-02823-9. BMC Med. 2023. PMID: 36978059 Free PMC article.
References
-
- Li T, Saldanha IJ, Jap J, Smith BT, Canner J, Hutfless SM, et al. A randomized trial provided new evidence on the accuracy and efficiency of traditional vs. electronically annotated abstraction approaches in systematic reviews. J Clin Epidemiol. 2019;115:77–89. doi: 10.1016/j.jclinepi.2019.07.005. - DOI - PubMed
MeSH terms
LinkOut - more resources
Full Text Sources