|
This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.
Please go to www.ahrq.gov for current information.
What Works and Why? Lessons Learned From Six AHRQ Grantees Implementing Comparative Effectiveness Research
This Webinar summarized findings from an evaluation of six AHRQ-funded grant projects seeking to promote and disseminate comparative effectiveness research findings. The six projects—three focusing on mental health care and three focusing on physical health care—were evaluated to determine the extent of successful dissemination and implementation of these findings in additional physician practices or States. Lessons learned from each of the grantee sites were presented as part of this Webinar.
Slide 1: What Works and Why? Lessons Learned from 6 AHRQ Grantees* Implementing Comparative Effectiveness Research
November 17, 2014
* AHRQ's Accelerating Implementation of Comparative Effectiveness Findings on Clinical and Delivery System Interventions by Leveraging AHRQ Networks program.
Slide 2: Welcome AHRQ Grantee Teams
2010: 6 teams funded under Accelerating Implementation of Comparative Effectiveness Findings on Clinical and Delivery System Interventions by Leveraging AHRQ Networks.
2014: 8 teams funded under Disseminating Patient Centered Outcomes Research to Improve Healthcare Delivery.
Slide 3: Overview of Today's Webinar
- High Level Overview of 6 Grants to Implement CER
(Moderator: Peggy McNamara, AHRQ)
- Pittsburgh Regional Health Initiative.
- Dartmouth CO-OP Project/CECH PBRNs.
- Rutgers University.
- Carolinas Healthcare System.
- Westat, working with University of Oklahoma Health Sciences Center.
- Johns Hopkins University.
- Grantee Lessons Learned
(Moderator: James Dearing, Michigan State University).
- Grantee Tools & Wrap-Up
(Peggy McNamara and Jan De La Mare, AHRQ).
Your questions and perspectives solicited throughout the Webinar.
Slide 4: Mental Health Interventions
- Pittsburgh Regional Health Initiative.
- Dartmouth CO-OP Project/CECH PBRNs.
- Rutgers University.
Slide 5: Partners in Integrated Care (PIC): Support Screening for Depression and Alcohol and Drug Misuse
- Funded Organization: Pittsburgh Regional Health Initiative (PRHI).
- Affiliated Network: AHRQ Chartered Value Exchange (CVE) Learning Network, and Network for Regional Healthcare Improvement (NRHI).
- Geographic Focus: Pennsylvania, Wisconsin, Minnesota, and Massachusetts.
Slide 6: PIC: Support Screening for Depression and Alcohol and Drug Misuse: Intervention at a Glance
Evidence Being Disseminated
- IMPACT model for USPSTF recommendation of adult depression screening when staff-assisted depression care supports are in place.
- SBIRT model for USPSTF recommendation of adult alcohol misuse screening and brief intervention.
- Adult screening and brief interventions for other drug misuse based on emerging and promising evidence at the time.
Primary Aim
- To rapidly disseminate and test an integrated IMPACT-SBIRT team-based model in 90 primary care office practices in 4 states, comparing response and remission rates to evidence base.
Intervention Description
- Planning: Collaborative creation of "PIC Toolkit", training modules, data tracking tools, software, and marketing materials.
- Implementation: Practice recruitment, staff hiring, and protocol-based multi-state roll-out.
- Refinement: Live (site-visits, regional meetings) and online (webinars, Tomorrow's HealthCare).
Lead Implementers
- "Integration Specialists"—LCSW, MSW, LPC, MA, or RN, trained in PIC protocol and motivational interviewing.
- Primary care physicians.
- Consulting psychiatrists.
External Supports
- Regional pay-for-performance.
- PCMH.
- ACO incentives.
- CEU credits.
Slide 7: PIC: Support Screening for Depression and Alcohol and Drug Misuse
Achievements
- Implementation in 57 primary care offices.
- Over 60,000 screens for depression and unhealthy substance use.
- Among eligible patients, 3,186 received depression services and 429 received substance misuse services.
- Response and remission outcomes comparable across sites (Kaplan-Meier survival curves).
- Defined role of office-based "Integration Specialist."
Sustainability
- PRHI Web site: All PIC materials available free of charge (PIC Toolkit, literature review, marketing videos).
- New grants: COMPASS (CMS Innovation Center); applications from WIPHL and PRHI/NRHI.
- State-based public health: Massachusetts (MA-SBIRT), Minnesota Department of Public Safety (SBIRT), University of Wisconsin (PIC).
- Managed care organizations: Medicaid payers (Pennsylvania).
- Employer outreaches: Wisconsin.
Slide 8: Teen Mental Health (TMH) Project
- Funded Organization: Dartmouth Primary Care Cooperative (Dartmouth CO-OP) Information Project.
- Affiliated Network: Clinicians Enhancing Child Health (CECH) Practice-Based Research Network (PBRN).
- Geographic Focus: Vermont, New Hampshire.
Slide 9: Teen Mental Health (TMH) Project: Intervention at a Glance
Evidence Being Disseminated
- USPSTF recommendation: "Screen adolescents (12–18 years of age) for major depressive disorder when systems are in place to ensure 1) accurate diagnosis, 2) psychotherapy, and 3) follow-up."
Primary Aim
- To develop, pilot test, implement, and evaluate a "TMH office system" in 15 rural pediatric and family medicine practices.
Intervention Description
- TMH office system to support identification and appropriate treatment of adolescent depression.
- In person and web-based training in TMH and customization support.
Lead Implementers
- Physicians and care managers in the practices.
External Supports
- 7 of 12 practices used TMH for American Board of Pediatrics' Maintenance of Certification.
- Option of using TMH to fulfill ACO and PCMH requirements.
- Marginal revenue from billing for screening did not cover billing costs for some practices.
- Disseminators engaged payers in discussions, but a financing model was not found to cover the full cost of the intervention.
Slide 10: Teen Mental Health (TMH) Project
Achievements
- TMH office system was developed and implemented in 12 practices.
- Total of 9,894 teens were screened.
Sustainability
- 7 of 12 practices continued to screen and use the TMH registry 9 months after active intervention ended.
- The remaining 5 practices report continuing some components, but did stop use of the registry.
Slide 11: Accelerating Utilization of CE Findings in Medicaid Mental Health: MEDNET
- Funded Organization: Rutgers University, Center for Health Services Research on Pharmacotherapy, Chronic Disease Management, and Outcomes.
- Affiliated Network: Multistate MEDNET/SMINET consortium of states, evolving from CERTs collaborative project with Medicaid Medical Directors Learning Network.
- Geographic Focus:
- 6 states: MO, TX, CA (Orange County), ME, OK, and WA.
- 2 additional "affiliate" states: OH and WI.
Slide 12: Medicaid Mental Health: MEDNET Intervention at a Glance
Evidence Being Disseminated
- 10 specific clinical target areas included: 1) mental health drug polypharmacy; 2) management of metabolic risks of antipsychotics (AP); 3) AP utilization for children under age 6; 4) higher than recommended doses of APs; 5) utilization of appropriate mental health services as a complement or alternative to psychotropic treatments; 6) consistency between diagnoses and treatments; 7) improvement of treatment adherence for adults with severe mental illness; 8) identification and prediction of high risk/high utilization beneficiaries and targeting of services to these subpopulations; 9) safe and effective psychotropic and mental health services use by foster care youth; and 10) safe and effective psychotropic and mental health services use by nursing home residents.
- Quality improvement (QI) science was also disseminated.
Primary Aim
- Safe and judicious prescribing of AP medications across life span and among multiple high risk Medicaid populations.
Intervention Description
- Multi-state public/academic consortium charged with sharing evidence-based practices and quality data at the provider level to improve AP prescribing practices.
- Standardized, consensus driven AP Quality Metrics were developed and used to: track improvements; identify individuals at risk, and benchmark trends among participating states.
- Leveraging local policy environments, customized state-specific CQI interventions were developed and implemented to improve prescribing patterns.
Lead Implementers
- State Medicaid, mental health and child welfare agency leadership.
- Experts from the Rutgers-Columbia mental health CERTs.
- Broad state level Stakeholder Collaborative workgroups.
External Supports
- Ongoing clinical, administrative, policy, and QI science TA was available throughout the project.
Slide 13: Medicaid Mental Health: MEDNET
Achievements
- 3 MEDNET Quality Metrics adapted and adopted by NCQA for inclusion in HEDIS 2015.
- Reviewed and improved Texas Foster Care Prescribing Parameters.
- State-specific prescribing oversight improvements:
- Community Mental Health Center feedback reports and CQI collaborative in WA.
- Managed Care Organization/county guideline development and metabolic management initiative on Orange County.
- Integration of quality metrics into MO and ME health home programs.
- New prescribing guidelines and oversight/second opinion triggers in TX.
- Integration of metrics into Medicaid Drug Utilization Review and care integration initiative in OK.
- Prescribing changes included 45% in reduction in AP polypharmacy in targeted WA clinics.
- 52% polypharmacy reduction in TX, and 41% in OK.
- Substantial reductions in AP treatment in youngest children in TX.
- Successful, continuation "EBP spread funding" from AHRQ–SMINET.
- Incorporation of measurement driven QI processes and metrics into ongoing state administrative processes, including managed care oversight.
Sustainability
- 6 of 6 participant states integrated the MEDNET quality metrics, often in multiple projects, to track quality improvements
- ACA Health Homes, External Quality Review Organization performance contracts, Drug Utilization Review programs, Prior Authorization algorithms, and CMS Dual Demonstration projects.
Slide 14: Physical Health Interventions
- Carolinas Health Care System.
- Westat, on behalf of University of Oklahoma Health Sciences Center.
- Johns Hopkins University.
Slide 15: Asthma Comparative Effectiveness (ACE): Using Shared Decision Making (SDM) to Improve Outcomes for Patients with Asthma within an Integrated Healthcare System
- Funded Organization: Carolinas Healthcare System.
- Affiliated Network: Mecklenburg Area Primary Care Research Network (MAPPR) Practice-Based Research Network (PBRN).
- Geographic Focus: North and South Carolina.
Slide 16: Asthma Comparative Effectiveness (ACE): Using SDM: Intervention at a Glance
Evidence Being Disseminated
- NHLBI guidelines for the diagnosis and treatment of Asthma and Shared Treatment Decision Making Improves Adherence and Outcomes in Poorly Controlled Asthma. Am J Respir Crit Care Med. March 15, 2010.
Primary Aim
- To adapt, implement, and evaluate a "Shared Decision Making Toolkit" in 6 primary care hospital-based practices serving a vulnerable Medicare population.
Intervention Description
- A facilitator-led approach was used to partner with providers, staff and train health coaches to adopt and implement the shared decision making toolkit; and assisted health coaches to convene clinics for patients.
Lead Implementers
- Facilitators train practice implementation team during a participatory "rollout" process adapted to culture at each practice. Typical implementers are physician champions and practice managers.
External Supports
- Physicians and practices used asthma SDM clinic for American Board of Family Medicine Maintenance of Certification.
- SDM was used to help 1 practice get PCMH level III certification.
- Hospital based practices can bill for support staff such as pharmacist acting as health coach for patients.
Slide 17: Asthma Comparative Effectiveness (ACE): Using SDM
Achievements
- 258 English & Spanish, Adult, and Pediatric patients participated in 358 SDM Half-Day Clinic Visits.
- At 6 months post SDM, asthma exacerbations dropped by 42% in the ED, 50% for hospitalizations, and a 46% drop in oral prednisone use.
- 86% of patients reported the asthma treatment decision was shared between the patient and provider.
Sustainability
- 4 of 6 original practices still hold regular SDM half day clinics and all 6 practices still use SDM toolkit components during asthma treatment planning.
- Results from this project were used to successfully obtain PCORI funding for state-wide dissemination of SDM across 30 sites.
- Partnership with Medicaid has enabled the toolkit to be incorporated in materials distributed to 278 practices across NC.
Slide 18: Leveraging Practice Based Research Networks (PBRNs) to Accelerate Implementation and Diffusion of Chronic Kidney Disease (CKD) Guidelines in Primary Care Practice
- Funded Organization: University of Oklahoma Health Sciences Center (OUHSC), Oklahoma City, OK.
- Affiliated Networks/Geographic Focus:
- Oklahoma Physicians Resource/Research Network.
- Los Angeles Practice-Based Research Network.
- Minnesota Academy of Family Physicians Research Network.
- Wisconsin Research and Education Network.
Slide 19: Chronic Kidney Disease (CKD) Project: Intervention at a Glance
Evidence Being Disseminated
- 8 processes of care from the National Kidney Foundation Kidney Disease Outcomes Quality Initiative Guidelines: 1) making and documenting the diagnosis, 2) ordering appropriate tests based upon severity of disease, 3) discontinuing potentially harmful medications, 4) starting potentially beneficial medications, 5) managing diabetes and cardiovascular disease risk factors, 6) educating patients about vein preservation, 7) giving appropriate immunizations, and 8) referring patients with advanced disease.
Primary Aim
- To determine whether networks could increase dissemination, implementation, and diffusion of evidence-based treatment guidelines for chronic kidney disease (CKD) by leveraging early adopter practices.
Intervention Description
- Wave I: Practices from 4 PBRNs received baseline and periodic performance feedback, academic detailing, and weekly practice facilitation for 6 months.
- Wave II: Wave I practices + 2 additional practices received similar performance feedback and academic detailing, monthly practice facilitation, and participated in 6 monthly "local" learning collaboratives (LLCs) led by Wave I clinicians.
Lead Implementers
- Physicians in the practices and Practice Facilitators provided by the PBRNs.
External Supports
- Maintenance Of Certification Part IV credit provided by the American Boards of Family Medicine and Internal Medicine.
- CME credit available through the American Academy of Family Physicians.
Slide 20: Chronic Kidney Disease (CKD) Project
Achievements
- Wave I: 711 patients, 31 of 32 practices:
- Increased use of angiotensin-converting enzyme inhibitors or angiotensin receptor blockers (ACEIs/ARBs), discontinuation of nonsteroidal anti-inflammatory drugs (NSAIDs), testing for anemia, and testing and/or treatment for vitamin D deficiency.
- Wave II: 1179 patients, 58 of 62 practices enrolled:
- Also increased use of ACEIs/ARBs and testing and/or treatment of vitamin D deficiency.
Sustainability
- Newly implemented processes were typically accompanied by the creation of templates, order sets, and other modifications to the electronic health records.
- Increased awareness of the importance of eGFR (an estimated value derived from a measured serum creatinine and population means for age and gender) created by the project
- Automatic calculation of eGFR by labs used by participating practices prior to or as a result of the project.
Slide 21: Cardiovascular Surgical Translational Study
- Funded Organization: Johns Hopkins Medicine Armstrong Institute for Patient Safety and Quality.
- Affiliated Network: None.
- Geographic Focus: 11 hospitals in 9 states (OH, FL, MA, MD, MO, NC, NY, SC, TX).
Slide 22: Cardiovascular Surgical Translational Study: Intervention at a Glance
Evidence Being Disseminated
- Prevention methods for 3 common infections.
Primary Aim
- Prevention methods for 3 common infections.
- To implement and evaluate the impact of a patient safety program in 11 of cardiac surgery operating rooms (OR), 8 intensive care units (ICU), 9 inpatient units, and 3 universal bed units on:
- Healthcare-associated infection (HAI) rates.
- Patient safety culture, cross-unit teamwork, and transitions of care.
- Comparison to passive audit and feedback.
Intervention Description
- Evidenced-based prevention toolkits for surgical site infections (SSI), central line-associated bloodstream infections (CLABSI) and ventilator-associated pneumonia or events (VAP/VAE) plus the Comprehensive Unit-based Safety Program (CUSP).
- Standardizing technical components of the work (evidence summaries and measures), while encouraging local innovation in the adaptive components (implementation and culture change).
Lead Implementers
- Unit-based multidisciplinary teams in 11 hospitals.
External Supports
Slide 23: Cardiovascular Surgical Translational Study
Achievements
- Development and implementation of SSI; CLABSI; and VAP/VAE prevention bundles and CUSP in 11 hospitals.
- Preliminary data (March 2014) show median CLABSI rates decreased to 0.00 per 1,000 line-days (baseline: 1.52, 1.23); median VAP rates sustained at 0.00 per 1,000 vent-days (baseline: 0.00); downward trend observed in median SSI rates but some variation exists.
- Culture regarding leadership improved; scores on open communication and feedback about errors were lower (1 yr. post-implementation vs. baseline).
Sustainability (Note: this project ended in July 2014; sustainability data not yet available)
- Embed: Make policies and procedures, train new people, walk the process.
- Expand: Pass it on to other units, identify and address the next challenge.
Slide 24: Grantee Discussion: Lessons Learned
What did you do that worked well?
What did you try that didn't work?
What are your funding, research and policy recommendations for AHRQ and other funders re: dissemination and implementation?
Facilitator: James Dearing, PhD
Michigan State University
Slide 25: Grantee Tools
Tools developed by grantee teams that may be useful for other dissemination and implementation teams to adapt.
Slide 26: PIC: Support Screening for Depression and Alcohol and Drug Misuse: Dissemination and Implementation Resources
Pittsburgh Regional Health Initiative
| Name of Tool |
Purpose of Tool |
Link to Download Tool |
1. PIC Toolkit
- Marketing and Communication.
- Train the Trainer.
- Primary Care Training.
- Practice Support & Care Delivery.
- IT & Measurement.
|
Entire suite of materials to conduct recruitment, implementation, and refinement |
http://www.prhi.org/initiatives/pic/pic-toolkit |
2.Marketing Videos
- Overview.
- The Pennsylvania Experience.
- The Minnesota Experience.
- The Business Case for PIC.
|
Quick orientation to project, recruitment, and stakeholder engagement |
http://www.prhi.org/initiatives/pic |
Slide 27: Teen Mental Health (TMH) Project
Dissemination and Implementation Resources
Dartmouth CO-OP Project/CECH PBRNs
Slide 28: Teen Mental Health (TMH) Project: Dissemination and Implementation Resources
Dartmouth CO-OP Project/CECH PBRNs
Slide 29: Medicaid Mental Health: MEDNET Dissemination and Implementation Resources
Rutgers University
Slide 30: Medicaid Mental Health: MEDNET Dissemination and Implementation Resources
Rutgers University
Slide 31: Medicaid Mental Health: MEDNET Dissemination and Implementation Resources
Rutgers University
Slide 32: Asthma Comparative Effectiveness (ACE): Using SDM Dissemination and Implementation Resources
Carolinas Healthcare System
| Name of Tool |
Purpose of Tool |
Link to Download Tool |
| 1. Asthma Shared Decision Making Toolkit |
- Shared Decision Making toolkit in English and Spanish.
- SDM engagement testimony from patents, health coaches and physicians.
- Implementation Resources.
|
asthma.carolinashealthcare.org/ |
| 2. Asthma Action Plan incorporating medication decision support based on NHBLI guidelines |
Asthma Action Plan generator |
https://asthma.carolinashealthcare.org/Form |
| 3. The Amazings asthma App game |
App to engage children in game that educates around asthma |
asthma.carolinashealthcare.org/ |
Slide 33: Chronic Kidney Disease (CKD) Project
Dissemination and Implementation Resources
Westat, working with University of Oklahoma Health Sciences Center.
| Name of Tool |
Purpose of Tool |
Link to Download Tool |
| 1. Chronic Kidney Disease Clinical Practice Guidelines Implementation Toolkit |
- CKD Guideline Recommendations for Primary Care (one-page).
- Screening and diagnosis guidelines.
- Patient education tasks and approaches.
- CKD: A patient's guide (English/Spanish).
- Medication discontinuation and cautions.
- Lab tests.
- Reducing rate of CKD complications.
- Immunizations.
- cardiovascular risk factor management.
- Nephrology referral.
|
Under development (Estimated date of completion: Spring 2015; contact Zsolt Nagykaldi at zsolt-nagykaldi@ouhsc.edu) |
| 2. Practice Facilitator Certificate Program |
Designed and developed by national experts who drew upon the training modules in the AHRQ Practice Facilitation Handbook. Web based with live video seminars schedules weekly and local internships at certified sites |
http://www.millardfillmorecollege.com/practice_facilitator_ad |
Slide 34: Cardiovascular Surgical Translational Study Dissemination and Implementation Resources
Johns Hopkins University
Slide 35: Cardiovascular Surgical Translational Study Dissemination and Implementation Resources
Johns Hopkins University
Page last reviewed January 2015
Page originally created January 2015
Internet Citation: What Works and Why? Lessons Learned From Six AHRQ Grantees Implementing Comparative Effectiveness Research. Content last reviewed January 2015. Agency for Healthcare Research and Quality, Rockville, MD. https://archive.ahrq.gov/research/findings/final-reports/whatworks/index.html | |
5600 Fishers Lane Rockville, MD 20857