|
This information is for reference purposes only. It was current when produced and may now be outdated. Archive material is no longer maintained, and some links may not work. Persons with disabilities having difficulty accessing this information should contact us at: https://info.ahrq.gov. Let us know the nature of the problem, the Web address of what you want, and your contact information.
Please go to www.ahrq.gov for current information.
Evaluation of ARRA Comparative Effectiveness Research Dissemination Contract Efforts
Appendix Q: Clinician Survey Longitudinal Results
Appendix Q contains the detailed results tables for the longitudinal analysis of all of the main outcome variables from the clinician survey (waves 1 and 2). This is a supplement to the findings reported in Section 6 of the report. The results tables are organized according to the four main outcomes: (1) awareness, (2) knowledge and understanding, (3) attitudes/perceived benefits, and (4) behavior/use.
For each outcome variable we analyzed, we tested whether there was a statistically significant increase between survey waves (wave 2 minus wave 1) at the p<0.05 level using a one-sided test (go to Section 2.5.2 Analytic Methodology of Clinician Survey details). The tables include key information such as: (1) wave-specific point estimates; (2) two-sided 95% confidence intervals for each wave-specific point estimate; (3) the estimate for the differences between survey waves, e.g., "Diff (2-1)"; and (4) one-sided 95% confidence intervals for the difference estimates.
Exhibit Q.1. Longitudinal Findings of Clinician Awareness of EHC Program, CER, and AHRQ
Aided awareness of EHC Program
| Wave |
N |
Proportion Aware |
Std. Err. |
Z-value |
P-value1 |
95% Conf. Interval |
| 1 |
1657 |
0.083887 |
0.00681 |
|
|
(0.070539, 0.097234) |
| 2 |
1558 |
0.121951 |
0.00829 |
|
|
(0.105703, 0.1382) |
| Diff (2-1) |
|
0.038065 |
0.010729 |
|
|
(0.02, 1) |
| |
|
under Ho: |
0.010696 |
3.56 |
0.0002 |
|
Aided awareness of CER
| Wave |
N |
Proportion Aware |
Std. Err. |
Z-value |
P-value1 |
95% Conf. Interval |
| 1 |
1623 |
0.181762 |
0.009573 |
|
|
(0.163, 0.200524) |
| 2 |
1480 |
0.196622 |
0.010331 |
|
|
(0.176373, 0.21687) |
| Diff (2-1) |
|
0.01486 |
0.014084 |
|
|
(-0.008, 1) |
| |
|
under Ho: |
0.014067 |
1.06 |
0.1454 |
|
Aided awareness of AHRQ
| Wave |
N |
Proportion Aware |
Std. Err. |
Z-value |
P-value1 |
95% Conf. Interval |
| 1 |
1669 |
0.32834 |
0.011495 |
|
|
(0.305811, 0.35087) |
| 2 |
1564 |
0.378517 |
0.012264 |
|
|
(0.354479, 0.402554) |
| Diff (2-1) |
|
0.050176 |
0.016809 |
|
|
(0.023, 1) |
| |
|
under Ho: |
0.016815 |
2.98 |
0.0014 |
|
Aided awareness of PCOR
| Wave |
N |
Proportion Aware |
Std. Err. |
Z-value |
P-value1 |
95% Conf. Interval |
| 1 |
1640 |
0.462805 |
0.012312 |
|
|
(0.438673, 0.486937) |
| 2 |
1491 |
0.488263 |
0.012945 |
|
|
(0.462891, 0.513635) |
| Diff (2-1) |
|
0.025458 |
0.017866 |
|
|
(-0.004, 1) |
| |
|
under Ho: |
0.017869 |
1.42 |
0.0771 |
|
1. One sided p-value using normal approximation to the binomial. Tests null hypothesis of no difference in wave 2 and wave1 proportions versus the alternative that wave 2 is bigger.
Exhibit Q.2. Longitudinal Findings of Clinician Knowledge and Understanding
CER Knowledge Scale1
| Wave |
N |
Mean |
Std. Err. |
T-value |
P-value1 |
95% Conf. Interval |
| 1 |
731 |
5.372093 |
0.091368 |
|
|
(5.192718, 5.551468) |
| 2 |
746 |
5.226542 |
0.094076 |
|
|
(5.041856, 5.411227) |
| Diff (2-1) |
|
-0.14555 |
0.131143 |
-1.1099 |
0.866 |
(-0.361, 11) |
EHC Program Knowledge Scale2
| Wave |
N |
Mean |
Std. Err. |
T-value |
P-value1 |
95% Conf. Interval |
| 1 |
134 |
6.395522 |
0.249794 |
|
|
(5.90144, 6.889605) |
| 2 |
183 |
6.322404 |
0.222036 |
|
|
(5.88431, 6.760499) |
| Diff (2-1) |
|
-0.07312 |
0.334211 |
-0.2188 |
0.587 |
(-0.625, 11) |
1. Among respondents aware of CER.
2. Among respondents aware of EHC Program.
Exhibit Q.3. Longitudinal Findings of Clinician Attitudes and Perceived Benefits
Interest in learning more about CER
| Wave |
N |
Mean |
Std. Err. |
T-value |
P-value |
95% Conf. Interval |
| 1 |
1659 |
3.570826 |
0.023526 |
|
|
(3.524681, 3.61697) |
| 2 |
1556 |
3.582262 |
0.025444 |
|
|
(3.532353, 3.632171) |
| Diff (2-1) |
|
0.011436 |
0.034654 |
0.33 |
0.371 |
(-0.046, 4) |
Interest in learning more about EHC Program
| Wave |
N |
Mean |
Std. Err. |
T-value |
P-value |
95% Conf. Interval |
| 1 |
1675 |
3.613134 |
0.022891 |
|
|
(3.568236, 3.658033) |
| 2 |
1569 |
3.639261 |
0.024839 |
|
|
(3.59054, 3.687981) |
| Diff (2-1) |
|
0.026126 |
0.033778 |
0.7735 |
0.22 |
(-0.029, 4) |
CER Perceived Benefits Scale1
| Wave |
N |
Mean |
Std. Err. |
T-value |
P-value |
95% Conf. Interval |
| 1 |
721 |
0.582683 |
0.019771 |
|
|
(0.543866, 0.621499) |
| 2 |
743 |
0.556072 |
0.01948 |
|
|
(0.517829, 0.594314) |
| Diff (2-1) |
|
-0.02661 |
0.027756 |
-0.9588 |
0.831 |
(-0.072, 4) |
CER Opinions Scale1
| Wave |
N |
Mean |
Std. Err. |
T-value |
P-value |
95% Conf. Interval |
| 1 |
705 |
3.68589 |
0.022094 |
|
|
(3.642512, 3.729268) |
| 2 |
731 |
3.727835 |
0.021696 |
|
|
(3.685242, 3.770429) |
| Diff (2-1) |
|
0.041945 |
0.030965 |
1.3546 |
0.088 |
(-0.009, 4) |
1. Among respondents aware of CER.
Exhibit Q.4. Longitudinal Findings of Clinician Behavior Change and Use
Ever visited EHC Program Web site2
| Wave |
N |
Proportion Visited |
Std. Err. |
Z-value |
P-value1 |
95% Conf. Interval |
| 1 |
291 |
0.278351 |
0.026273 |
|
|
(0.226856, 0.329845) |
| 2 |
252 |
0.412698 |
0.031013 |
|
|
(0.351914, 0.473483) |
| Diff (2-1) |
|
0.134348 |
0.040646 |
|
|
(0.067, 1) |
| |
|
under Ho: |
0.040783 |
3.29 |
0.0005 |
|
Likely to use EHC Program patient summaries within the next year
| Wave |
N |
Proportion |
Std. Err. |
Z-value |
P-value1 |
95% Conf. Interval |
| 1 |
1665 |
0.465466 |
0.012224 |
|
|
(0.441506, 0.489425) |
| 2 |
1563 |
0.474088 |
0.01263 |
|
|
(0.449334, 0.498843) |
| Diff (2-1) |
|
0.008623 |
0.017577 |
|
|
(-0.02, 1) |
| |
|
under Ho: |
0.017577 |
0.49 |
0.3119 |
|
Recently used EHCP clinician products3
| Wave |
N |
Proportion |
Std. Err. |
Z-value |
P-value1 |
95% Conf. Interval |
| 1 |
135 |
0.511111 |
0.043023 |
|
|
(0.426789, 0.595434) |
| 2 |
185 |
0.437838 |
0.036476 |
|
|
(0.366347, 0.509329) |
| Diff (2-1) |
|
-0.07327 |
0.056404 |
-1.3 |
0.9027 |
(-0.166, 1) |
Recently shared consumer summaries with patients3
| Wave |
N |
Proportion |
Std. Err. |
Z-value |
P-value1 |
95% Conf. Interval |
| 1 |
136 |
0.352941 |
0.040978 |
|
|
(0.272625, 0.433257) |
| 2 |
185 |
0.318919 |
0.034265 |
|
|
(0.25176, 0.386078) |
| Diff (2-1) |
|
-0.03402 |
0.053417 |
-0.64 |
0.7386 |
(-0.122, 1) |
1. One sided p-value using normal approximation to the binomial. Tests null hypothesis of no difference in wave 2 and wave1 proportions versus the alternative that wave 2 is bigger.
2. Among respondents who had heard of the EHC Program Web site.
3. Among respondents aware of EHC Program.
Return to Contents
Page last reviewed October 2013
Page originally created December 2014
Internet Citation: Evaluation of ARRA Comparative Effectiveness Research Dissemination Contract Efforts. Content last reviewed October 2013. Agency for Healthcare Research and Quality, Rockville, MD. https://archive.ahrq.gov/research/findings/final-reports/arracer/arracer-apq.html | |
5600 Fishers Lane Rockville, MD 20857