A comparison of statistical methods for identifying out-of-date systematic reviews

Download poster as PDF

Download poster as PDF

Abstract text
Background:Systematic reviews (SRs) can provide accurate and reliable evidence, typically about the effectiveness of health interventions. Evidence is dynamic, and if SRs are out-of-date this information may not be useful; it may even be harmful.

Objectives: To compare five statistical methods in terms of agreement and practical in identifying out-of-date SR

Methods:A retrospective cohort of SRs registered in the Cochrane Pregnancy and Childbirth Group (CPCG), published between 2008 and 2010, were considered for inclusion. For each eligible CPCG review, data were extracted and “3-years previous” meta-analyses were assessed for the need to update, given the data from the most recent 3 years. Each of the five statistical methods were used, with random effects analyses throughout the study.

Results:80 reviews were included in this study; most were in the area of induction of labour. The numbers of reviews identified as being out-of-date using the Ottawa, recursive cumulative meta-analysis (CMA), and Barrowman methods were 34, 7, and 7 respectively. The overall agreement among the three discriminating statistical methods was slight (Kappa=0.14; 95% CI 0.05 to 0.23). The recursive CMA method was found to be a practical in terms of data requirements and ease of calculation. No reviews were identified as being out-of-date using the simulation-based power method, or the CMA for sufficiency and stability method.

Conclusions:The recursive cumulative meta-analysis method should be a practical method to examine the need to update SRs.
Pattanittum P1, Laopaiboon M1, Moher D2, Lumbiganon P3, Ngamjarus C1
1 Thai Cochrane Network, Thailand, AND Department of Biostatistics and Demography, Faculty of Public Health, Khon Kaen University, Khon Kaen, Thailand
2 Clinical Epidemiology Program, Ottawa Hospital Research Institute, AND Department of Epidemiology and Community Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada
3 Thai Cochrane Network, Thailand, AND Department of Obstetrics and Gynaecology, Faculty of Medicine, Khon Kaen University, Khon Kaen, Thailand
Presenting author and contact person
Presenting author: 
Porjai Pattanittum
Contact person Affiliation Country
Porjai Pattanittum (Contact this person) Biostatistics and Demography Department, Public Health Faculty, Khon Kaen University, Khon Kaen, Thailand 40002 Thailand