Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 Jun 25;20(6):e10507.
doi: 10.2196/10507.

Interactive Visual Displays for Interpreting the Results of Clinical Trials: Formative Evaluation With Case Vignettes

Jiantao Bian  1 , Charlene Weir  1   2 , Prasad Unni  1   3 , Damian Borbolla  1 , Thomas Reese  1 , Yik-Ki Jacob Wan  1 , Guilherme Del Fiol  1
Affiliations

Interactive Visual Displays for Interpreting the Results of Clinical Trials: Formative Evaluation With Case Vignettes

Jiantao Bian et al. J Med Internet Res. .

Abstract

Background: At the point of care, evidence from randomized controlled trials (RCTs) is underutilized in helping clinicians meet their information needs.

Objective: To design interactive visual displays to help clinicians interpret and compare the results of relevant RCTs for the management of a specific patient, and to conduct a formative evaluation with physicians comparing interactive visual versus narrative displays.

Methods: We followed a user-centered and iterative design process succeeded by development of information display prototypes as a Web-based application. We then used a within-subjects design with 20 participants (8 attendings and 12 residents) to evaluate the usability and problem-solving impact of the information displays. We compared subjects' perceptions of the interactive visual displays versus narrative abstracts.

Results: The resulting interactive visual displays present RCT results side-by-side according to the Population, Intervention, Comparison, and Outcome (PICO) framework. Study participants completed 19 usability tasks in 3 to 11 seconds with a success rate of 78% to 100%. Participants favored the interactive visual displays over narrative abstracts according to perceived efficiency, effectiveness, effort, user experience and preference (all P values <.001).

Conclusions: When interpreting and applying RCT findings to case vignettes, physicians preferred interactive graphical and PICO-framework-based information displays that enable direct comparison of the results from multiple RCTs compared to the traditional narrative and study-centered format. Future studies should investigate the use of interactive visual displays to support clinical decision making in care settings and their effect on clinician and patient outcomes.

Keywords: clinical decision-making; clinician information needs; information display; information foraging theory; information seeking behavior.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: None declared.

Figures

Figure 1
Figure 1
Visualization of study structure.
Figure 2
Figure 2
Article list table with trials on various treatments for diabetes mellitus. This display is the landing page of the information displays. It provides a table with the title, patient population, and study arms of each study. The goal is to allow clinicians to quickly scan each study and select relevant ones for further review.
Figure 3
Figure 3
Comparison table display with four trials on various treatments for diabetes mellitus. This display contains key elements of selected studies in a tabular format according to the Population, Intervention, Comparison, Outcomes (PICO) framework [29-32]. Studies are displayed in columns, and attributes of studies are displayed in rows. Study results for primary outcomes and adverse events are represented in bar graphs [46]. Hovering over a bar brings up a callout with details on the intervention of the selected study arm. The scale of each measure is normalized across all studies to enable direct visual comparison. An illustration of the comparison table display for randomized controlled trials on rheumatoid arthritis is available in Multimedia Appendix 1, Figure A2.
Figure 4
Figure 4
Efficacy graph display (top) and side effects graph display (bottom) with four trials on various treatments for diabetes mellitus. These two displays provide graphical comparisons of study primary outcomes and adverse effects respectively. Users can choose to set the bar graph for a specific outcome measure, overall adverse effects, or the most common adverse effect across all the arms of the selected studies.
Figure 5
Figure 5
Mean differences for participants’ perceived efficiency, effectiveness, effort, user experience and preference of interactive visual displays versus narrative abstracts.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Del Fiol G, Workman T, Gorman P. Clinical questions raised by clinicians at the point of care: a systematic review. JAMA Intern Med. 2014 May;174(5):710–8. doi: 10.1001/jamainternmed.2014.368.1846630 - DOI - PubMed
    1. Leape L, Bates D, Cullen D, Cooper J, Demonaco H, Gallivan T, Hallisey R, Ives J, Laird N, Laffel G. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA. 1995 Jul 05;274(1):35–43. - PubMed
    1. Maviglia S, Yoon C, Bates D, Kuperman G. KnowledgeLink: impact of context-sensitive information retrieval on clinicians' information needs. J Am Med Inform Assoc. 2006;13(1):67–73. doi: 10.1197/jamia.M1861. http://europepmc.org/abstract/MED/16221942 M1861 - DOI - PMC - PubMed
    1. Ioannidis J. Why most published research findings are false. PLoS Med. 2005 Aug;2(8):e124. doi: 10.1371/journal.pmed.0020124. http://dx.plos.org/10.1371/journal.pmed.0020124 04-PLME-E-0321R2 - DOI - DOI - PMC - PubMed
    1. Cook D, Sorensen K, Hersh W, Berger R, Wilkinson J. Features of effective medical knowledge resources to support point of care learning: a focus group study. PLoS One. 2013;8(11):e80318. doi: 10.1371/journal.pone.0080318. http://dx.plos.org/10.1371/journal.pone.0080318 PONE-D-13-33090 - DOI - DOI - PMC - PubMed

Publication types