NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.
Defining the Benefits of Stakeholder Engagement in Systematic Reviews
Research White Papers
Investigators: Erika Cottrell, PhD, MPP, Evelyn Whitlock, MD, MPH, Elisabeth Kato, MD, MRP, Stacey Uhl, MS, Suzanne Belinson, MPH, PhD, Christine Chang, MD, MPH, Ties Hoomans, PhD, David Meltzer, MD, PhD, Hussein Noorani, MS, Karen Robinson, PhD, Karen Schoelles, MD, SM, FACP, Makalapua Motu'apuaka, BS, Johanna Anderson, MPH, Robin Paynter, MLIS, and Jeanne-Marie Guise, MD, MPH.
Author Information and AffiliationsStructured Abstract
Background:
Stakeholder engagement efforts in the Evidence-based Practice Center (EPC) program initially focused on defining opportunities and developing materials to involve stakeholders in systematic reviews. Over time, the basic mechanics of working with stakeholders have become a routine part of the systematic review process, allowing the program to begin to explore how to improve stakeholder engagement and make it more effective
Purpose:
The purpose of this research paper is to examine the benefits and challenges of engaging stakeholders in the process of developing and performing systematic reviews, drawing upon findings from the literature and Key Informant (KI) interviews with program leaders, systematic reviewers and stakeholders from within the EPC program and other international evidence-based programs.
Methods:
We searched a range of databases, including Ovid MEDLINE, EBM Reviews, and Scopus. A gray literature search identified relevant guidance issued by internationally recognized organizations, and additional citations were identified both via research team members and by pearling of relevant article bibliographies. Abstracts and full text articles were reviewed for relevance by three investigators. In addition to the literature search we conducted a series of interviews with those who know the most about this process: 60 KIs were invited and one investigator facilitated all of the interviews. With permission from all participants we recorded and transcribed all calls; two investigators and two research assistants conducted analysis using NVivo to identify and synthesize recurring themes. In addition to identifying overarching themes, we conducted a more detailed exploratory analysis of the KI interviews with an eye towards articulating the benefits of engaging specific types of stakeholders at each phase of the systematic review.
Results:
Of the 299 abstracts and 80 full-text articles reviewed, 24 addressed in some fashion the benefits, challenges, measurement or evaluation of stakeholder engagement. Benefits cited included identifying and prioritizing topics for research; providing pragmatic feedback on the research protocol; aiding in recruitment of research participants; helping the researchers understand the research subject's perspective; ensuring that findings are interpreted with the end user in mind and that final products are readable and accessible; and facilitating wider dissemination and uptake of research findings. There was almost no discussion of measurement or evaluation of the impact of stakeholder engagement. Of the 60 KIs we invited, 34 agreed to participate, we conducted 12 discussion sessions (60 to 90 minutes per session) with between one and four participants each. Indeed, it was not uncommon for stakeholders to represent more than one perspective. Overarching themes from our KI interviews were organized according to the three guiding questions of this white paper: (1) What are the potential or expected benefits of involving stakeholders in systematic reviews? (2) What are the challenges of involving stakeholders in systematic reviews? (3) How can we measure the impact of stakeholder engagement in systematic reviews?
Conclusions/Recommendations:
Although it is recognized by many as an important next step, to date there have been few efforts to measure the benefits/tradeoffs of specific stakeholder engagement processes or differing approaches to selecting and engaging differing stakeholder types. In order to refine our processes for efficiently and effectively engaging stakeholders, we need to develop methods to evaluate the impact of stakeholder engagement based on a more concrete understanding of the specific benefits we are hoping to achieve. Toward this end, we reviewed the existing literature and conducted a series of KI interviews in an effort to more explicitly define the expected benefits of engaging stakeholders in systematic reviews.
Contents
- Preface
- Introduction
- Methods
- Results
- Discussion
- Conclusions and Future Directions
- References
- Appendix A Search Strategy
- Appendix B Relevant Information From Literature Search
- Appendix C Interview Guides
- Appendix D Select Quotes From Key Informant Discussions
- Appendix E Key Information Perspectives on Benefits of Specific Types of Stakeholder Engagement in the Topic Refinement Phase
- Appendix F Key Informants
Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services1, Contract No. 290-2012-00004-C. Prepared by: Scientific Resource Center, Portland, OR
Suggested citation:
Cottrell E, Whitlock E, Kato E, Uhl S, Belinson S, Chang C, Hoomans T, Meltzer D, Noorani H, Robinson K, Schoelles K, Motu'apuaka M, Anderson J, Paynter R, Guise JM. Defining the Benefits of Stakeholder Engagement in Systematic Reviews. Research White Paper. (Prepared by the Scientific Resource Center under Contract No. 290-2012-00004-C.) AHRQ Publication No. 14-EHC006-EF. Rockville, MD: Agency for Healthcare Research and Quality. March 2014. www.effectivehealthcare.ahrq.gov/reports/final.cfm.
This report is based on research conducted by the Agency for Healthcare Research and Quality (AHRQ) Scientific Resource Center Working Group. The findings and conclusions in this document are those of the authors, who are responsible for its contents; the findings and conclusions do not necessarily represent the views of AHRQ. Therefore, no statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.
The information in this report is intended to help health care decisionmakers—patients and clinicians, health system leaders, and policymakers, among others—make well-informed decisions and thereby improve the quality of health care services. This report is not intended to be a substitute for the application of clinical judgment. Anyone who makes decisions concerning the provision of clinical care should consider this report in the same way as any medical reference and in conjunction with all other pertinent information (i.e., in the context of available resources and circumstances presented by individual patients).
This report may be used, in whole or in part, as the basis for development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.
None of the investigators have any affiliations or financial involvement that conflicts with the material presented in this report.
- 1
540 Gaither Road, Rockville, MD 20850; www
.ahrq.gov
- NLM CatalogRelated NLM Catalog Entries
- Review Discerning the Perception and Impact of Patients Involved in Evidence-based Practice Center Key Informant Interviews[ 2017]Review Discerning the Perception and Impact of Patients Involved in Evidence-based Practice Center Key Informant InterviewsCornman DH, White CM. 2017 Sep
- Review EPC Methods: AHRQ End-User Perspectives of Rapid Reviews[ 2016]Review EPC Methods: AHRQ End-User Perspectives of Rapid ReviewsHartling L, Guise JM, Hempel S, Featherstone R, Mitchell MD, Motu'apuaka ML, Robinson KA, Schoelles K, Totten A, Whitlock E, et al. 2016 Apr
- PROTOCOL: Barriers and facilitators to stakeholder engagement in health guideline development: A qualitative evidence synthesis.[Campbell Syst Rev. 2022]PROTOCOL: Barriers and facilitators to stakeholder engagement in health guideline development: A qualitative evidence synthesis.Magwood O, Riddle A, Petkovic J, Lytvyn L, Khabsa J, Atwere P, Akl EA, Campbell P, Welch V, Smith M, et al. Campbell Syst Rev. 2022 Jun; 18(2):e1237. Epub 2022 Apr 25.
- Review Comparing Methods to Make Research More Patient Centered[ 2019]Review Comparing Methods to Make Research More Patient CenteredWilkins C, Stallings SC, Villalta-Gil V, Houston M, Vaughn Y, Richmond A, Novak L, Joosten Y, Simpson C, Israel T, et al. 2019 Dec
- Review Understanding Health-Systems' Use of and Need for Evidence To Inform Decisionmaking[ 2017]Review Understanding Health-Systems' Use of and Need for Evidence To Inform DecisionmakingWhite CM, Sanders Schmidler GD, Butler M, Wang Z, Robinson K, Mitchell MD, Berkman N, Henderson J, Fiordalisi C, Hartling L, et al. 2017 Oct
- Defining the Benefits of Stakeholder Engagement in Systematic ReviewsDefining the Benefits of Stakeholder Engagement in Systematic Reviews
- Contextual Frameworks for Research on the Implementation of Complex System Inter...Contextual Frameworks for Research on the Implementation of Complex System Interventions
- Assessment of a Method to Detect Signals for Updating Systematic ReviewsAssessment of a Method to Detect Signals for Updating Systematic Reviews
- EPC Methods: An Exploration of Methods and Context for the Production of Rapid R...EPC Methods: An Exploration of Methods and Context for the Production of Rapid Reviews
- Comparison of Effects as Evidence Evolves From Single Trials to High-Quality Bod...Comparison of Effects as Evidence Evolves From Single Trials to High-Quality Bodies of Evidence
Your browsing activity is empty.
Activity recording is turned off.
See more...