Towards Modernizing the Systematic Review Pipeline: Efficient Updating via Data Mining

Abstract text
Background: It is difficult to keep up with new information for both performing new reviews and updating existing reviews. Indeed, it is estimated that more than half of the systematic reviews in the Cochrane Library have not been updated for at least two years. One reason for this is the exponential growth of the biomedical literature; there is simply too much literature to keep up.

Objectives: To demonstrate that modern data mining tools can be one step in reducing the labor necessary to produce and maintain systematic reviews.

Methods: We used four continuously updated, manually-curated resources that summarize MEDLINE-indexed articles in entire fields using systematic review methods (PDGene, AlzGene, SzGene for genetic determinants of Parkinson’s, Alzheimer’s and schizophrenia, respectively; and the Tufts CEARegistry for cost-effectiveness analyses). In each dataset, we trained a classification model on citations screened up to 2009. We then evaluated its ability to classify citations published in 2010 as “relevant” or “irrelevant”, using human screening as the gold standard.

Results: Classification models did not miss any of the 104, 65 and 179 eligible citations in PDGene, AlzGene and SzGene, respectively, and missed only one out of 79 in CEA Registry (100% sensitivity for the three first, 99% for the fourth). The respective specificities were 90%, 93%, 90% and 73%. Had the semi-automated system been used in 2010, a human would have needed to read only 605/5616 citations to update the PDGene registry (11%) and 555/7298=8 percent, 717/5381=13% and 334/1015=33% for the other three databases.

Conclusions: Data mining methodologies can reduce the burden of updating systematic reviews, without missing more papers than humans.
wallace B1, small K2, brodley C3, lau J1, schmid C1, bertram L4, lill C5, cohen J2, trikalinos T1
1 Brown, USA
2 Tufts Medical Center, USA
3 Tufts, USA
4 Max Planck Institute for Molecular Genetics, Germany
5 Johannes Gutenberg University, Germany
Presenting author and contact person
Presenting author: 
Christopher Schmid
Contact person Affiliation Country
byron wallace (Contact this person) Brown USA
Date and Location
Oral session A5O3
Monday 1 October 2012 - 11:40 - 12:00