An empirical assessment of bivariate methods for meta-analysis of test performance

Abstract text
Background: Meta-analysis of diagnostic test accuracy studies must account for correlation between sensitivity and specificity. Summary receiver operating characteristic (SROC) curves and bivariate models have become methods of choice. However, different forms of these techniques exist and methods based on univariate models are still in common use.

Objectives: Empirically compare different approaches to analyzing sensitivity and specificity using a large dataset of published diagnostic test meta-analyses.

Methods: We searched PUBMED (1987-2003) for meta-analyses that reported 2×2 tables for at least one diagnostic outcome. Methods evaluated included: fixed effect univariate meta-analyses; univariate and bivariate random effects meta-analyses with maximum likelihood (ML) using both normal approximation and exact binomial likelihood. The bivariate model with exact binomial likelihood was also fit using a fully Bayesian approach. We constructed SROC curves using the Moses-Littenberg fixed effects method (weighted and unweighted), the Rutter-Gatsonis hierarchical SROC (HSROC) method, and four alternative HSROC methods.

Results: We found 308 meta-analyses. All normal approximation methods estimated summary sensitivity and specificity closer to 0.5 and gave smaller standard errors compared to exact binomial likelihood methods. Marginal results of univariate and bivariate random effects meta-analyses were similar regardless of the estimation method. Bivariate models using ML and fully Bayesian methods gave similar point estimates but Bayesian models indicated additional uncertainty. All bivariate methods poorly estimated the correlation between sensitivity and specificity. Moses-Littenberg and Rutter-Gatsonis SROC curves produced similar results. Alternative parameterizations of the HSROC regression resulted in markedly different summary lines in one-third of the meta-analyses; this depended on the estimated covariance between sensitivity and specificity.

Conclusions: Meta-analytic summaries of sensitivity and specificity can be sensitive to model choice when information is sparse, particularly when the correlation between sensitivity and specificity is large and poorly estimated. In such cases, Bayesian bivariate methods with exact likelihoods can appropriately quantify estimation uncertainty.
Dahabreh IJ1, Trikalinos TA2, Lau J1, Schmid CH3
1 Center for Clinical Evidence Synthesis, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, Boston, MA
2 Center for Evidence-based Medicine, Program in Public Health, Brown University, Providence, RI
3 Biostatistics Research Center, Institute for Clinical Research and Health Policy Studies, Tufts Medical Center, Boston, MA
Presenting author and contact person
Presenting author: 
Christopher Schmid
Contact person Affiliation Country
Issa Dahabreh (Contact this person) Tufts Medical Center USA
Date and Location
Oral session B6O4
Tuesday 2 October 2012 - 12:00 - 12:20