Systematic review identifies six metrics and one method for assessing literature search effectiveness but no consensus on appropriate use
- PMID: 29526555
- DOI: 10.1016/j.jclinepi.2018.02.025
Systematic review identifies six metrics and one method for assessing literature search effectiveness but no consensus on appropriate use
Abstract
Objectives: To identify the metrics or methods used by researchers to determine the effectiveness of literature searching where supplementary search methods are compared to bibliographic database searching. We also aimed to determine which metrics or methods are summative or formative and how researchers defined effectiveness in their studies.
Study design and setting: Systematic review. We searched MEDLINE and Embase to identify published studies evaluating literature search effectiveness in health or allied topics.
Results: Fifty studies met full-text inclusion criteria. Six metrics (sensitivity, specificity, precision, accuracy, number needed to read, and yield) and one method (capture recapture) were identified.
Conclusion: Studies evaluating effectiveness need to identify clearly the threshold at which they will define effectiveness and how the evaluation they report relates to this threshold. Studies that attempt to investigate literature search effectiveness should be informed by the reporting of confidence intervals, which aids interpretation of uncertainty around the result, and the search methods used to derive effectiveness estimates should be clearly reported and validated in studies.
Keywords: Effectiveness; Literature searching; Sensitivity; Systematic searching.
Copyright © 2018 Elsevier Inc. All rights reserved.
Similar articles
-
A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: a methodological review.Syst Rev. 2017 Nov 28;6(1):234. doi: 10.1186/s13643-017-0625-1. Syst Rev. 2017. PMID: 29179733 Free PMC article. Review.
-
A comparison of the performance of seven key bibliographic databases in identifying all relevant systematic reviews of interventions for hypertension.Syst Rev. 2016 Feb 9;5:27. doi: 10.1186/s13643-016-0197-5. Syst Rev. 2016. PMID: 26862061 Free PMC article.
-
Locating qualitative studies in dementia on MEDLINE, EMBASE, CINAHL, and PsycINFO: A comparison of search strategies.Res Synth Methods. 2018 Dec;9(4):579-586. doi: 10.1002/jrsm.1280. Epub 2017 Nov 27. Res Synth Methods. 2018. PMID: 29080334
-
No consensus exists on search reporting methods for systematic reviews.J Clin Epidemiol. 2008 Aug;61(8):748-54. doi: 10.1016/j.jclinepi.2007.10.009. Epub 2008 Feb 14. J Clin Epidemiol. 2008. PMID: 18586178
-
A review of the reporting of web searching to identify studies for Cochrane systematic reviews.Res Synth Methods. 2018 Mar;9(1):89-99. doi: 10.1002/jrsm.1275. Epub 2017 Nov 9. Res Synth Methods. 2018. PMID: 29065246 Review.
Cited by
-
Precision and Recall of Search Strategies for Identifying Studies on Work-Related Psychosocial Risk Factors in PubMed.J Occup Rehabil. 2023 Dec;33(4):776-784. doi: 10.1007/s10926-023-10110-w. Epub 2023 Mar 21. J Occup Rehabil. 2023. PMID: 36941513
-
Development and evaluation of a search filter to identify prognostic factor studies in Ovid MEDLINE.BMC Med Res Methodol. 2022 Apr 10;22(1):107. doi: 10.1186/s12874-022-01595-9. BMC Med Res Methodol. 2022. PMID: 35399050 Free PMC article.
-
Mapping Evidence in Teaching Palliative Care in Undergraduate Curriculum of Healthcare Professionals Qualification: A Systematic Scoping Review Protocol.Res Sq [Preprint]. 2024 Jul 3:rs.3.rs-4313706. doi: 10.21203/rs.3.rs-4313706/v1. Res Sq. 2024. PMID: 39011104 Free PMC article. Preprint.
-
Literature searching methods or guidance and their application to public health topics: A narrative review.Health Info Libr J. 2022 Mar;39(1):6-21. doi: 10.1111/hir.12414. Epub 2021 Dec 1. Health Info Libr J. 2022. PMID: 34850535 Free PMC article. Review.
-
Educational interventions to improve literature searching skills in the health sciences: a scoping review.J Med Libr Assoc. 2020 Oct 1;108(4):534-546. doi: 10.5195/jmla.2020.954. J Med Libr Assoc. 2020. PMID: 33013210 Free PMC article.
Publication types
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources
Miscellaneous
