How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis

1. 

   View larger version:

   • In this window
   • In a new window

   Figure 1. Includes the search protocols to identify candidate new trials, application of criteria from the original review to identify eligible new trials, meta-analytic pooling of new results with previous meta-analytic results, and identification of new systematic reviews on the same topic or “pivotal trials” (published in 1 of the 5 highest-impact general medical journals or more than 3 times the sample size of the previous largest trial) that met any of our criteria for qualitative signals for updating. An individual reviewer reached a tentative conclusion about the presence of quantitative and qualitative signals for updating, but each review was discussed in detail by the project team to reach a final consensus decision. For reviews that did not have any signals for updating, the group also decided whether the searches had been adequate or whether more comprehensive searching for new trials might be required, including more detailed electronic searching and hand-searching of for new trials relevant to the original review. Overall process for determining updating status.ACP Journal Club
2. 

   View larger version:

   • In this window
   • In a new window

   Appendix Figure 1. R = reason; RCT = randomized, controlled trial. Screening of potential systematic reviews for inclusion in cohort.
3. 

   View larger version:

   • In this window
   • In a new window

   Figure 2. The immediate decrease in survival at time zero reflects the 7 systematic reviews for which signals for updating had already occurred at the time of publication. The low number of reviews at risk after 10 years reflects the fact that the sample spanned 1995 to 2005 and censoring occurred on 1 September 2006. Thus, only reviews published before September 1996 and having no signals for updating could have more than 10 years of observation. Overall survival time (95% CI) free of signals for updating.
4. 

   View larger version:

   • In this window
   • In a new window

   Appendix Figure 2. Stratified by cardiovascular reviews (  = 20) versus reviews on all other topics (  = 80). Survival of the original systematic review by clinical topic area.nn
5. 

   View larger version:

   • In this window
   • In a new window

   Appendix Figure 3. Ratio of new total sample size to old total sample size is >2, which occurred for 25% of systematic reviews in the cohort. Kaplan–Meier plot showing the effect on survival of increasing the total number of patients by more than a factor of 2.