Introduction: PAIN⁺ and PubMed are two electronic databases with two different mechanisms of evidence retrieval. PubMed is used to "Pull" evidence where clinicians can enter search terms to find answers while PAIN⁺ is a newly developed evidence repository where along with "Pull" service there is a "Push" service that alerts users about new research and the associated quality ratings, based on the individual preferences for content and altering criteria.

Purpose: The primary purpose of the study was to compare yield and usefulness of PubMed and PAIN⁺ in retrieving evidence to address clinical research questions on pain management. The secondary purpose of the study was to identify what search terms and methods were used by clinicians to target pain research.

Study design: Two-phase double blinded randomized crossover trial.

Methods: Clinicians (n = 76) who were exposed to PAIN⁺ for at least 1 year took part in this study. Participants were required to search for evidence 2 clinical question scenarios independently. The first clinical question was provided to all participants and thus, was multi-disciplinary. Participants were randomly assigned to search for evidence on their clinical question using either PAIN⁺ or PubMed through the electronic interface. Upon completion of the search with one search engine, they were crossed over to the other search engine. A similar process was done for a second scenario that was discipline-specific. The yield was calculated using number of retrieved articles presented to participants and usefulness was evaluated using a series of Likert scale questions embedded in the testing.

Results: Multidisciplinary scenario: Overall, the participants had an overall one-page yield of 715 articles for PAIN⁺ and 1135 articles for PubMed. The topmost article retrieved by PAIN⁺ was rated as more useful (p = 0.001). While, the topmost article retrieved by PubMed was rated as consistent with current clinical practice (p = 0.02). PubMed (48%) was preferred over PAIN⁺ (39%) to perform multidisciplinary search (p = 0.02). Discipline specific scenario: The participants had an overall one-page yield of 1046 articles for PAIN⁺ and 1398 articles for PubMed. The topmost article retrieved by PAIN⁺ was rated as more useful (p = 0.001) and consistent with current clinical practice (p = 0.02) than the articles retrieved by PubMed. PAIN⁺ (52%) was preferred over PubMed (29%) to perform discipline specific search.

Conclusion: Clinicians from different disciplines find both PAIN⁺ and PubMed useful for retrieving research studies to address clinical questions about pain management. Greater preferences and perceived usefulness of the top 3 retrieved papers was observed for PAIN⁺, but other dimensions of usefulness did not consistently favor either search engine.

Trial registration: Registered with ClinicalTrials.gov Identifier: NCT01348802 , Date: May 5, 2011.