Aims: To analyse the effect on systematic reviews in diabetes interventions of including only trials that are indexed in medline, and to assess the impact of adding trials from other databases and the grey literature.

Methods: All systematic reviews of diabetes interventions which included a meta-analysis of randomized controlled trials, and were published since 1996, were selected. The impact on the meta-analysis of including only those trials indexed in medline, and the effect of then adding trials from other sources, was assessed. Where possible this was measured quantitatively, by redoing the meta-analysis, otherwise a qualitative estimate was made.

Results: Forty-four systematic reviews met our inclusion criteria. There were 120 articles reporting trial data which were not indexed in medline. These came from 52% of the reviews. In 34% of the reviews, basing a meta-analysis on a search of only medline would miss trials that could affect the result. Sources of non-medline data which had the biggest effect on the meta-analyses were journal articles from central and embase (mainly in Diabetes, Nutrition and Metabolism) and unpublished data (mainly from industry). The exceptions were journal articles on herbal medicine, mostly indexed in Chinese language databases.

Conclusions: A search of only the medline database is insufficient for systematic reviews of diabetes, because in about 34% of reviews the missed trials could affect the results of the meta-analysis. It is recommended that central (on the Cochrane Library) also be searched. Scanning meeting abstracts, and seeking unpublished data are also recommended if the intervention has only recently been introduced.