Research Briefs

Balas, E.A., Li, Z.R., Spencer, D.C., and others (1996, January/February). "An expert system for performance-based direct delivery of published clinical evidence." (AHCPR grant HS07715). Journal of the American Medical Informatics Association 3(1), pp. 56-65.

The authors describe development of the Quality Feedback Expert System (QFES), a tool for implementing clinical practice guidelines. It is made up of three databases, supports integration of recommendations from several guidelines into a comprehensive and measurable quality improvement plan, analyzes actual practice patterns and compares them with accepted recommendations, and generates a confidential report to those who significantly deviate from clinical recommendations. The researchers demonstrated the feasibility of the practice pattern analysis by the QFES in a sample of 182 urinary tract infection cases from a primary care clinic, and found that in a set of clinical activities, certain questions/ procedures were associated with significant and unexplained variations. They conclude that the QFES is a flexible tool for implementing clinical guidelines in diverse and changing clinical areas without the need for special program development.

Cassard, S.D., Patrick, D.L., Damiano, A.M., and others. (1995, December). "Reproducibility and responsiveness of the VF-14: An index of functional impairment in patients with cataracts." (AHCPR grant HS06280). Archives of Ophthalmology 113, pp. 1508-1513.

This study by the Cataract Patient Outcomes Research Team (PORT) assesses the test-retest reliability and responsiveness of the VF-14, which is an index of functional impairment in patients with cataracts. They assessed the VF-14 in 552 patients 4 months after they underwent cataract surgery in one eye and a subset of these who did not subsequently undergo surgery for the second eye by the 1-year followup. The VF-14 was reproducible in stable patients during an 8-month period, and it was three times more responsive to clinically significant changes in vision than the Sickness Impact Profile, a generic health status measure. Responsiveness of theVF-14 was greater in patients who reported greater problems with vision before surgery.

Cohen, S.B., and Carlson, B.L. (1995). "Characteristics of reluctant respondents in the National Medical Expenditure Survey." Journal of Economic and Social Measurement 21, pp. 269-296.

This paper analyzes the factors which distinguished those households that required special efforts to obtain their participation in the Household Component of the 1987 National Medical Expenditure Survey (NMES). The NMES provides national and regional estimates of the health care use, expenditures, sources of payment, and health insurance coverage of the U.S. civilian noninstitutionalized population. Researchers from the Agency for Health Care Policy and Research examined the quality of the data obtained from these reluctant respondents and assessed the likelihood of achieving their cooperation for all required rounds of data collection. The results have significant implications for design of future medical expenditure surveys. Reprints (AHCPR Publication No. 96-R052) are available from AHCPR.

Iezzoni, L.I., Shwartz, M., Ash, A.S., and others (1996). "Severity measurement methods and judging hospital death rates for pneumonia." (AHCPR grant HS06742). Medical Care 34(1), pp. 11-28.

Payers and policymakers are increasingly examining hospital mortality rates as indicators of hospital quality of care. However, this study shows that perceptions of individual hospital mortality rates vary, depending on the severity of illness measures used to adjust actual mortality with expected mortality. The researchers examined 14 severity of illness measurement methods to judge hospital death rates for pneumonia at 105 acute care hospitals nationwide. After adjusting for age, sex, diagnosis, and severity of illness, 73 hospitals had observed mortality rates that did not differ significantly from expected rates according to all 14 severity methods; two had rates significantly higher than expected for all 14 severity methods. For 30 hospitals, observed mortality rates differed significantly from expected rates when judged by one or more, but not all 14 severity methods. The 14 severity methods agreed about relative hospital performance more often than expected by chance, but perceptions of individual hospitals' mortality rates varied when different severity adjustment methods were used for almost one-third of facilities.

Katz, J.N., Chang, L.C., Sangha, O., and others (1996). "Can comorbidity be measured by questionnaire rather than medical record review?" (NRSA fellowship F32 HS00040). Medical Care 34(1), pp. 73-84.

The number and severity of coexisting medical conditions (comorbidity) of patients explains much of the variation in clinical and economic outcomes of patients with the same disease. This study explores whether comorbidity can be measured by questionnaire rather than medical record review. The researchers developed a brief comorbidity questionnaire that included items corresponding to each element of the medical record-based Charlson index and administered it to 170 inpatients. Although the correlation between comorbidity measures was weaker in less educated patients, the authors conclude that a questionnaire version of the Charlson index is reproducible, valid, and offers practical advantages over medical record-based assessments.

Katz, J.N., Punnett, L., Simmons, B.P., and others (1996, January). "Workers' compensation recipients with carpal tunnel syndrome: The validity of self-reported health measures." (AHCPR grant HS06813). American Journal of Public Health 86(1), pp. 52-56.

This study compares the reliability, validity, and responsiveness of self-reported measures of health-related quality of life in recipients and non-recipients of workers' compensation who have carpal tunnel syndrome. Patients with carpal tunnel syndrome completed questionnaires at study enrollment and 6 months later; scales measuring symptom severity, functional status, and satisfaction were included. The internal consistencies for each scale were high and virtually identical in recipients and nonrecipients of workers' compensation. The correlations between self-reported and objectively measured grip strength were .32 in recipients and .30 in nonrecipients, regardless of whether the recipients were out of work. The researchers conclude that the reliability, validity, and responsiveness of these measures were comparable in nonrecipients and recipients of workers' compensation, and that these data support the use of self-report measures in studies of workers.

McDonald, C.J., Overhage, M., Tierney, W.M., and others. (1996, January). "The promise of computerized feedback systems for diabetes care." (AHCPR grant HS07719). Annals of Internal Medicine 124 (1 pt 2), pp. 170-174.

Most current medical uses of computer-based feedback control are open loop, where a human is interposed between the suggested intervention and the delivered treatment. Open-loop systems have already been used in diabetes care to suggest insulin dosage adjustments and treatments for hypercholes-terolemia and to remind physicians of various interventions to reduce the complications of diabetes mellitus. However, existing applications have only scratched the surface, according to these authors. Many more facets of diabetes management could be standardized and assisted by open-loop control systems if the management rules could be more exactly specified. New primary studies and decision analyses are needed to define the optimal use of some interventions.

Robinson, J.C., and Gardner, L.B. (1995). "Adverse selection among multiple competing health maintenance organizations." (AHCPR grant HS06815). Medical Care 33(12), pp. 1161-1175.

This study examines risk selection by nine health plans competing for 16,182 employees of one large firm in 1989: one conventional fee-for-service plan, one group-model health maintenance organization (HMO), and seven network and independent practice model HMOs. The researchers developed and compared measures of risk using weights based on HMO and fee-for-service expenditure data. Predicted annual expenditures per enrollee exhibited a 23 percent range from lowest (favorable selection) to highest (adverse selection) risk plans using the HMO weights and a 17 percent range using fee-for-service weights. The fee-for-service plan and group-model HMO with large enrollments had risk mixes near the center of the spectrum. Smaller HMOs exhibited the extreme forms of both favorable and adverse selection.

Spector, W.D. (1996). "Functional disability scales." In B. Spilker (ed.), Quality of Life and Pharmacoeconomics in Clinical Trials, Second Edition. Philadelphia: Lippincott-Raven Publishers, pp. 133-143.

The periodic assessment of disabilities has become an integral part of the standard medical evaluation of the elderly. This book chapter by William D. Spector, Ph.D., of the Center for Organization and Delivery Studies, Agency for Health Care Policy and Research, reviews a small number of functional disability scales that have received acceptance in the clinical and research arenas and have demonstrated sufficient reliability and validity. He emphasizes scales that include one type of disability measure, and points out that no scale is best for all purposes and that a scale should be chosen based on its specific purpose. Moreover, the properties of the scale should be carefully studied after implementation to assure that expected relationships between items exist. This is particularly important if the scale is applied to a new population or if modifications have been made. Dr. Spector cautions researchers about constructing new scales or using scales that have not been validated and about combining items in simplistic ways without doing appropriate scalability and validity analyses.

Tsevat, J., Solzan, J.G., Kuntz, K.M., and others (1996). "Health values of patients infected with human immunodeficiency virus: Relationship to mental health and physical functioning." (AHCPR grant HS06673) Medical Care 34(1), pp. 44-57.

According to this study, most AIDS patients are unwilling to trade any years of life, no matter how ill they are, for a shorter time in perfect health. The researchers used three health status measures and three health value measures to ask 139 patients infected with HIV about their health status and how they valued their health. Each participant was interviewed twice at 6-month intervals. As expected, the health status of HIV-infected patients who remained asymptomatic or remained symptomatic but did not develop AIDS changed little over 6 months, whereas health status deteriorated in patients with AIDS and those in whom HIV infection progressed. In contrast, health values, and in particular time-tradeoff scores, remained stable even in the face of changes in health status and disease progression. These findings suggest that either patients gradually acclimated to their deteriorating health state or they redefined their concept of "excellent health," conclude the researchers.

Zhou, Z.H. (1996, January-March). "Empirical Bayes combination of estimated (AHCPR grant HS08559). areas under ROC curves using estimating equations." Medical Decision Making 16, pp. 24-28.

Evaluating the accuracies of diagnostic tests in detecting the presence of disease is very important for both quality of care and cost containment. A receiving operating characteristics (ROC) curve allows the study of the inherent discrimination capability of a diagnostic test. The empirical Bayes (EB) method does not assume that individual studies all have the same true ROC area and provides a simple way to express study-level heterogeneity with a two-stage model. The author presents a synthesis of the EB method and the method of estimating equations to combine individual ROC area estimates from different studies of the same diagnostic test into a single estimate.

Return to Contents

AHCPR Publication No. 96-0055
Current as of April 1996