Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Aug;26(4):1896-1911.
doi: 10.1177/0962280215592269. Epub 2015 Jun 26.

Performance of methods for meta-analysis of diagnostic test accuracy with few studies or sparse data

Yemisi Takwoingi  1 , Boliang Guo  2 , Richard D Riley  3 , Jonathan J Deeks  1
Affiliations

Performance of methods for meta-analysis of diagnostic test accuracy with few studies or sparse data

Yemisi Takwoingi et al. Stat Methods Med Res. 2017 Aug.

Abstract

Hierarchical models such as the bivariate and hierarchical summary receiver operating characteristic (HSROC) models are recommended for meta-analysis of test accuracy studies. These models are challenging to fit when there are few studies and/or sparse data (for example zero cells in contingency tables due to studies reporting 100% sensitivity or specificity); the models may not converge, or give unreliable parameter estimates. Using simulation, we investigated the performance of seven hierarchical models incorporating increasing simplifications in scenarios designed to replicate realistic situations for meta-analysis of test accuracy studies. Performance of the models was assessed in terms of estimability (percentage of meta-analyses that successfully converged and percentage where the between study correlation was estimable), bias, mean square error and coverage of the 95% confidence intervals. Our results indicate that simpler hierarchical models are valid in situations with few studies or sparse data. For synthesis of sensitivity and specificity, univariate random effects logistic regression models are appropriate when a bivariate model cannot be fitted. Alternatively, an HSROC model that assumes a symmetric SROC curve (by excluding the shape parameter) can be used if the HSROC model is the chosen meta-analytic approach. In the absence of heterogeneity, fixed effect equivalent of the models can be applied.

Keywords: Diagnostic accuracy; HSROC model; bivariate model; diagnostic odds ratio; hierarchical models; meta-analysis; random effects; sensitivity; sparse data; specificity.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
Forest plot of sensitivity and specificity estimates from studies included in the two motivating examples. FN: false negative; FP: false positive; TN: true negative; TP: true positive.
Figure 2.
Figure 2.
Profile log-likelihood function of the covariance parameter in the bivariate model applied to the appendicitis example.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Reitsma JB, Glas AS, Rutjes AWS, et al. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol 2005; 58: 982–990. - PubMed
    1. Rutter CM, Gatsonis C. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med 2001; 20: 2865–2884. - PubMed
    1. Leeflang MMG, Deeks JJ, Gatsonis C, et al. Cochrane Diagnostic Test Accuracy Working Group. Systematic reviews of diagnostic test accuracy. Ann Intern Med 2008; 149: 889–897. - PMC - PubMed
    1. Harbord RM, Whiting P, Sterne JAC, et al. An empirical comparison of methods for meta-analysis of diagnostic accuracy showed hierarchical models are necessary. J Clin Epidemiol 2008; 61: 1095–1103. - PubMed
    1. Macaskill P, Gatsonis C, Deeks JJ, et al. Chapter 10: analysing and presenting results. In: Deeks JJ, Bossuyt PM and Gatsonis C (eds) Cochrane handbook for systematic reviews of diagnostic test accuracy. Version 1.0. The Cochrane Collaboration, 2010. http://srdta.cochrane.org/.

MeSH terms