Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 Mar 12;3(3):MR000041.
doi: 10.1002/14651858.MR000041.pub2.

Search strategies to identify observational studies in MEDLINE and Embase

Li Li  1 , Helen E Smith, Rifat Atun, Lorainne Tudor Car
Affiliations

Search strategies to identify observational studies in MEDLINE and Embase

Li Li et al. Cochrane Database Syst Rev. .

Abstract

Background: Systematic reviews are essential for decision-making. Systematic reviews on observational studies help answer research questions on aetiology, risk, prognosis, and frequency of rare outcomes or complications. However, identifying observational studies as part of systematic reviews efficiently is challenging due to poor and inconsistent indexing in literature databases. Search strategies that include a methodological filter focusing on study design of observational studies might be useful for improving the precision of the search performance.

Objectives: To assess the sensitivity and precision of a search strategy with a methodological filter to identify observational studies in MEDLINE and Embase.

Search methods: We searched MEDLINE (1946 to April 2018), Embase (1974 to April 2018), CINAHL (1937 to April 2018), the Cochrane Library (1992 to April 2018), Google Scholar and Open Grey in April 2018, and scanned reference lists of articles.

Selection criteria: Studies using a relative recall approach, i.e. comparing sensitivity or precision of a search strategy containing a methodological filter to identify observational studies in MEDLINE and Embase against a reference standard, or studies that compared two or more methodological filters.

Data collection and analysis: Two review authors independently screened articles, extracted relevant information and assessed the quality of the search strategies using the InterTASC Information Specialists' Sub-Group (ISSG) Search Filter Appraisal Checklist.

Main results: We identified two eligible studies reporting 18 methodological filters. All methodological filters in these two studies were developed using terms from the reference standard records.The first study evaluated six filters for retrieving observational studies of surgical interventions. The study reported on six filters: one Precision Terms Filter (comprising terms with higher precision while maximum sensitivity was maintained) and one Specificity Terms Filter (comprising terms with higher specificity while maximum sensitivity was maintained), both of which were adapted for MEDLINE, for Embase, and for combined MEDLINE/Embase searches. The study reported one reference standard consisting of 217 articles from one systematic review of which 83.9% of the included studies were case seriesThe second study reported on 12 filters for retrieving comparative non-randomised studies (cNRSs) including cohort, case-control, and cross-sectional studies. This study reported on 12 filters using four different approaches: Fixed method A (comprising of a fixed set of controlled vocabulary (CV) words), Fixed method B (comprising a fixed set of CV words and text words (TW)), Progressive method (CV) (a random choice of study design-related CV terms), and Progressive method (CV or TW) (a random choice of study design-related CV terms, and title and abstracts-based TWs). The study reported four reference standards consisting of 89 cNRSs from four systematic reviews.The six methodological filters developed from the first study reported sensitivity of 99.5% to 100% and precision of 16.7% to 21.1%. The Specificity Terms Filter for combined MEDLINE/Embase was preferred because it had higher precision and equal sensitivity to the Precision Terms Filter. The 12 filters from the second study reported lower sensitivity (48% to 100%) and much lower precision (0.09% to 4.47%). The Progressive method (CV or TW) had the highest sensitivity.There were methodological limitations in both included studies. The first study used one surgical intervention-focused systematic review thus limiting the generalizability of findings. The second study used four systematic reviews but with less than 100 studies. The external validation was performed only on Specificity Terms Filter from the first study Both studies were published 10 years ago and labelling and indexing of observational studies has changed since then.

Authors' conclusions: We found 18 methodological filters across two eligible studies. Search strategies from the first study had higher sensitivity and precision, underwent external validation and targeted observational studies. Search strategies from the second study had lower sensitivity and precision, focused on cNRSs, and were not validated externally. Given this limited and heterogeneous evidence, and its methodological limitations, further research and better indexation are needed.

PubMed Disclaimer

Conflict of interest statement

None.

Figures

1
1
PRISMA Study flow diagram
See this image and copyright information in PMC

Update of

  • doi: 10.1002/14651858.MR000041

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

References to studies included in this review

Fraser 2006 {published and unpublished data}
    1. Fraser C, Burr J. Identifying observational studies of surgical interventions in Medline and Embase: validation of a search filter [abstract]. IV Annual Meeting of Health Technology Assessment International. 2007 June:297.
    1. Fraser C, Murray A, Burr J. Identifying non‐randomised studies in MEDLINE and EMBASE [abstract]. Italian Journal of Public Health [Internet] 2005;2(Suppl 1):286.
    1. Fraser C, Murray A, Burr J. Identifying observational studies of surgical interventions in MEDLINE and EMBASE. BMC Medical Research Methodology 2006;6:41‐9. - PMC - PubMed
Furlan 2006 {published and unpublished data}
    1. Furlan AD, Bombardier C. Controlled vocabulary associated with non‐randomized studies of interventions for low back pain [abstract]. XI Cochrane Colloquium: Evidence, Health Care and Culture. 2003 Oct:69.
    1. Furlan AD, Irvin E, Bombardier C. Limited search strategies were effective in finding relevant non‐randomized studies. Journal of Clinical Epidemiology 2006;59(12):1303‐11. - PubMed

References to studies excluded from this review

Brown 2006 {published data only}
    1. Brown ML. New search strategies optimize MEDLINE retrieval of sound studies on treatment or prevention of health disorders. A review of: Haynes, R. Brian, K. Ann McKibbon, Nancy L. Wilczynski, Stephen D. Walter, and Stephen R. Were. “Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey.” BMJ 330.7501 (21 May 2005): 1179. Evidence Based Library and Information Practice 2006;1(2):23‐6. - PMC - PubMed
Fraser 1998 {published data only (unpublished sought but not used)}
    1. Fraser C, Thomson MA. Identifying non‐randomised studies in Medline [abstract]. Sixth International Cochrane Colloquium; Baltimore, MD, USA. 1998 Oct 22‐26.
Geersing 2012 {published data only}
    1. Geersing GJ, Bouwmeester W, Zuithoff P, Spijker R, Leeflang M, Moons KG. Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PLOS One 2012; Vol. 7, issue 2:e32844. - PMC - PubMed
Grimshaw 2007 {published data only}
    1. Grimshaw J, McGowan J, Salzwedel D. Update of the EPOC methodological search filter [abstract]. XV Cochrane Colloquium. 2007 Oct 23‐27:77‐78.
Haynes 1994 {published data only}
    1. Haynes RB, Wilczynski N, McKibbon KA, Walker CJ, Sinclair JC. Developing optimal search strategies for detecting clinically sound studies in MEDLINE. Journal of the American Medical Informatics Association 1994;1(6):447‐58. - PMC - PubMed
Haynes 2004 {published data only}
    1. Haynes RB, Wilczynski NL. Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey. BMJ 2004;328(7447):1040‐5. - PMC - PubMed
Haynes 2005A {published data only}
    1. Haynes RB, Kastner M, Wilczynski NL. Developing optimal search strategies for detecting clinically sound and relevant causation studies in EMBASE. BMC Medical Informatics and Decision Making 2005;5(1):8‐14. - PMC - PubMed
Haynes 2005B {published data only}
    1. Haynes RB, McKibbon KA, Wilczynski NL, Walter SD, Werre SR. Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey. BMJ 2005;330(7501):1179‐84. - PMC - PubMed
Holland 2005 {published data only}
    1. Holland JL, Wilczynski NL, Haynes RB, Hedges T. Optimal search strategies for identifying sound clinical prediction studies in EMBASE. BMC Medical Informatics and Decision Making 2005;5:11‐6. - PMC - PubMed
Montori 2005 {published data only}
    1. Montori VM, Wilczynski NL, Morgan D, Haynes RB, and for the Hedges Team. Optimal search strategies for retrieving systematic reviews from Medline: analytical survey. BMJ 2005;330(7482):68‐73. - PMC - PubMed
Mowatt 1999 {published data only}
    1. Mowatt G, Hero L, Campbell M, Fraser C, Grilli R, Grimshaw J, et al. Use of non‐randomised evidence in Cochrane Effective Practice and Organisation of Care (EPOC) group's reviews [abstract]. 7th Annual Cochrane Colloquium; Rome Italy [Internet]. 1999 Oct 5‐9.
Waffenschmidt 2017 {published data only}
    1. Waffenschmidt S, Hermanns T, Gerber‐Grote A, Mostardt S. No suitable precise or optimized epidemiologic search filters were available for bibliographic databases. Journal of Clinical Epidemiology 2017;82:112‐8. - PubMed
Wieland 2002 {published data only}
    1. Wieland S, Brodney S, Dickersin K. Designing an efficient and precise search strategy for observational studies. 10th Cochrane Colloquium Abstracts, Stavanger. 2002 31 July ‐ 3 August:23‐24.
Wieland 2005 {published data only}
    1. Wieland S, Dickersin K. Selective exposure reporting and Medline indexing limited the search sensitivity for observational studies of the adverse effects of oral contraceptives. Journal of Clinical Epidemiology 2005;58(6):560‐7. - PubMed
Wilczynski 1993 {published data only}
    1. Wilczynski NL, Walker CJ, McKibbon KA, Haynes RB. Assessment of methodologic search filters in MEDLINE. Proceedings of the Annual Symposium on Computer Application in Medical Care 1993;NA:601‐5. - PMC - PubMed
Wilczynski 2003 {published data only}
    1. Wilczynski NL, Haynes RB, Hedges T. [Developing optimal search strategies for detecting clinically sound causation studies in MEDLINE]. AMIA 2003 Symposium Proceedings. 2003:719‐23. - PMC - PubMed
Wilczynski 2004A {published data only}
    1. Wilczynski NL, Haynes RB. Developing optimal search strategies for detecting clinically sound prognostic studies in MEDLINE: an analytic survey. BMC Medicine 2004;2:23‐7. - PMC - PubMed
Wilczynski 2004B {published data only}
    1. Wilczynski N, Haynes B. Optimal search strategies for retrieving scientifically strong studies of treatment and diagnosis from MEDLINE: an analytical survey [abstract]. 12th Cochrane Colloquium: Bridging the Gaps; Ottawa, Ontario, Canada [Internet]. 2004 Oct 2‐6:70‐1.
Wilczynski 2004C {published data only}
    1. Wilczynski NL, Haynes RB, Lavis JN, Ramkissoonsingh R, Arnold‐Oatley AE. Optimal search strategies for detecting health services research studies in MEDLINE. Canadian Medical Association Journal 2004;171(10):1179‐85. - PMC - PubMed
Wilczynski 2005A {published data only}
    1. Wilczynski NL, Haynes RB. Optimal search strategies for detecting clinically sound prognostic studies in EMBASE: an analytic survey. Journal of the American Medical Informatics Association 2005;12(4):481‐5. - PMC - PubMed
Wilczynski 2005B {published data only}
    1. Wilczynski NL, Haynes RB. EMBASE search strategies for identifying methodologically sound diagnostic studies for use by clinicians and researchers. BMC Medicine 2005;3:7‐12. - PMC - PubMed
Wilczynski 2010 {published data only}
    1. Wilczynski NL, Haynes RB, for the QI Hedges Team. Optimal search filters for detecting quality improvement studies in Medline. Quality & Safety in Health Care 2010;19(6):e31‐5. - PubMed
Wong 2003 {published data only}
    1. Wong SSL, Wilczynski NL, Haynes RB, Ramkissoonsingh R. Developing optimal search strategies for detecting sound clinical prediction studies in MEDLINE. AMIA Annual Symposium Proceedings 2003;2003:728‐32. - PMC - PubMed

Additional references

Beynon 2013
    1. Beynon R, Leeflang MM, McDonald S, Eisinga A, Mitchell RL, Whiting P, et al. Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane Database of Systematic Reviews 2013, Issue 9. [DOI: 10.1002/14651858.MR000022.pub3] - DOI - PMC - PubMed
BMJ Best Practice 2018
    1. BMJ Best Practice. Study design filter. https://bestpractice.bmj.com/info/toolkit/learn‐ebm/study‐design‐search‐... 2018.
Doust 2005
    1. Doust JA, Pietrzak E, Sanders S, Glasziou PP. Identifying studies for systematic reviews of diagnostic tests was difficult due to the poor sensitivity and precision of methodologic filters and the lack of information in the abstract. Journal of Clinical Epidemiology 2005;58:444‐9. - PubMed
Durão 2015
    1. Durão S, Kredo T, Volmink J. Validation of a search strategy to identify nutrition trials in PubMed using the relative recall method. Journal of Clinical Epidemiology 2015;68(6):610‐6. - PubMed
Edwards 2002
    1. Edwards P, Clarke M, DiGuiseppi C, Pratap S, Roberts I, Wentz R. Identification of randomized controlled trials in systematic reviews: accuracy and reliability of screening records. Statistics in Medicine 2002;21(11):1635‐40. - PubMed
EndNote X7 [Computer program]
    1. Thomson Reuters. EndNote X7. New York, USA: Thomson Reuters Corporation, 2015.
Glanville 2008
    1. Glanville J, Bayliss S, Booth A, Dundar Y, Fernandes H, Fleeman ND, et al. So many filters, so little time: the development of a search filter appraisal checklist. Journal of the Medical Library Association 2008;96(4):356‐61. - PMC - PubMed
Glanville 2009
    1. Glanville J, Kaunelis D, Mensinkai S. How well do search filters perform in identifying economic evaluations in MEDLINE and EMBASE. International Journal of Technology Assessment in Health Care 2009;25(4):522‐9. - PubMed
Glasziou 2001
    1. Glasziou P, Irwig L, Bain C, Colditz G. Systematic Reviews in Health Care: a Practical Guide. Cambridge: Cambridge University Press, 2001.
Golder 2009
    1. Golder S, Loke YK. Search strategies to identify information on adverse effects: a systematic review. Journal of the Medical Library Association 2009;97(2):84‐92. - PMC - PubMed
Golder 2018
    1. Golder S, Wright K, Loke YK. The development of search filters for adverse effects of surgical interventions in Medline and Embase. Health Information and Libraries Journal 2018;35(2):121‐9. - PMC - PubMed
Gough 2012a
    1. Gough D, Oliver S, Thomas J. An Introduction to Systematic Reviews. London: SAGE, 2012.
Gough 2012b
    1. Gough D, Thomas J, Oliver S. Clarifying differences between review designs and methods. Systematic Reviews 2012;1:28. - PMC - PubMed
Hayward 1997
    1. Hayward S, Brunton G, Thomas K, Ciliska D. Searching for the evidence:source, time and yield. 2nd International Conference Scientific Basis of Health Services & 5th Annual Cochrane Colloquium. 1997.
Higgins 2011
    1. Higgins JP, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from www.cochrane‐handbook.org.
Horsley 2011
    1. Horsley T, Dingwall O, Sampson M. Checking reference lists to find additional studies for systematic reviews. Cochrane Database of Systematic Reviews 2011, Issue 8. [DOI: 10.1002/14651858.MR000026.pub2] - DOI - PMC - PubMed
ISSG 2017
    1. Glanville J, Lefebvre C, Wright K, editors. ISSG Search Filter Resource [Internet]. The InterTASC Information Specialists' Sub‐Group; 2008 [updated 2017 Aug 14]. https://sites.google.com/a/york.ac.uk/issg‐search‐filters‐resource/home 2017.
Jenkins 2004
    1. Jenkins M. Evaluation of methodological search filters‐a review. Health Information and Libraries Journal 2004;21(3):148‐63. - PubMed
Lefebvre 2011
    1. Lefebvre C, Manheimer E, Glanville J. Chapter 6: Searching for studies. In: Higgins JP, Green S, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration. Available from handbook.cochrane.org.
Lefebvre 2013
    1. Lefebvre C, Glanville J, Wieland LS, Coles B, Weightman AL. Methodological developments in searching for studies for systematic reviews: past, present and future?. Systematic Reviews 2013;2:78. - PMC - PubMed
Lefebvre 2017
    1. Lefebvre C, Glanville J, Beale S, Boachie C, Duffy S, Fraser C, et al. Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval:five literature reviews and a qualitative study. Health Technology Assessment 2017; Vol. 21, issue 69. - PMC - PubMed
Lemeshow 2005
    1. Lemeshow AR, Blum RE, Berlin JA, Stoto MA, Colditz GA. Searching one or two databases was insufficient for meta‐analysis of observational studies. Journal of Clinical Epidemiology 2005;58:867‐73. - PubMed
Ligthelm 2007
    1. Ligthelm RJ, Borzi V, Gumprecht J, Kawamori R, Wenying Y, Valensi P. Importance of observational studies in clinical practice. Clinical Therapeutics 2007;29:1284‐92. - PubMed
Littleton 2004
    1. Littleton D, Marsalis S, Bliss DZ. Searching the literature by design. Western Journal of Nursing Research 2004;26(8):891‐908. - PubMed
Manchikanti 2009
    1. Manchikanti L, Datta S, Smith HS, Hirsch JA. Evidence‐based medicine, systematic reviews, and guidelines in interventional pain management: part 6. Systematic reviews and meta analyses of observational studies. Pain Physician 2009;12:819‐50. - PubMed
McMaster Hedges Team 2017a
    1. McMaster Hedges Team. Search Filters for MEDLINE in Ovid Syntax and the PubMed translation. https://hiru.mcmaster.ca/hiru/HIRU_Hedges_MEDLINE_Strategies.aspx 2017.
McMaster Hedges Team 2017b
    1. McMaster Hedges Team. Search Strategies for EMBASE in Ovid Syntax. https://hiru.mcmaster.ca/hiru/HIRU_Hedges_EMBASE_Strategies.aspx 2017.
Robinson 2002
    1. Robinson KA, Dickersin K. Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed. International Journal of Epidemiology 2002;31(1):150‐3. - PubMed
Sampson 2006
    1. Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, et al. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Medical Research Methodology 2006;6:33. - PMC - PubMed
SIGN 2018
    1. Scottish Intercollegiate Guidelines Network. Search filter for observational studies. https://www.sign.ac.uk/search‐filters.html.
UTHealth 2018
    1. University of Texas School of Public Health. Search Filters for Various Databases: Ovid PsycINFO. http://libguides.sph.uth.tmc.edu/search_filters/ovid_psycinfo_filters 2018.
Vandenbroucke 2014
    1. Vandenbroucke JP, Elm E, Altman DG, Gøtzsche PC, Mulrow CD, Pocock SJ, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. International Journal of Surgery 2014;12(12):1500‐24. - PubMed
Viswanathan 2013
    1. Viswanathan M, Berkman ND, Dryden DM, Hartling L. Assessing Risk of Bias and Confounding in Observational Studies of Interventions or Exposures: Further Development of the RTI Item Bank. Methods Research Report.. Vol. AHRQ Publication No. 13‐EHC106‐EF, Rockville, MD: Agency for Healthcare Research and Quality, 2013. - PubMed
Wilczynski 2007
    1. Wilczynski NL, Haynes RB. EMBASE search strategies achieved high sensitivity and specificity for retrieving methodologically sound systematic reviews. Journal of Clinical Epidemiology 2007;60:29‐33. - PubMed
Young 2011
    1. Young T, Hopewell S. Methods for obtaining unpublished data. Cochrane Database of Systematic Reviews 2011, Issue 11. [DOI: 10.1002/14651858.MR000027.pub2] - DOI - PMC - PubMed
Yousefi‐Nooraie 2013
    1. Yousefi‐Nooraie R, Irani S, Mortaz‐Hedjri S, Shakiba B. Comparison of the efficacy of three PubMed search filters in finding randomized controlled trials to answer clinical questions. Journal of Evaluation in Clinical Practice 2013;19(5):723‐6. - PubMed

References to other published versions of this review

Marcano Belisario 2013
    1. Marcano Belisario JS, Tudor Car L, Reeves TJ, Gunn LH, Car J. Search strategies to identify observational studies in MEDLINE and EMBASE. Cochrane Database of Systematic Reviews 2013, Issue 12. [DOI: 10.1002/14651858.MR000041] - DOI - PMC - PubMed

Publication types

MeSH terms

LinkOut - more resources