The Science and Art of Deduction: Complex Systematic Overviews

Like all other arts, the science of deduction and analysis is one which can only be acquired by long and patient study … let the inquirer begin by mastering the more elementary problems.

—Sherlock Holmes, A Study in Scarlet

Organizations that develop clinical practice guidelines increasingly use systematic reviews to evaluate the benefits and harms associated with health care interventions (1). Well-executed systematic reviews efficiently synthesize evidence to answer focused clinical questions and can provide a strong foundation for clinical recommendations. Traditionally, systematic reviews build this foundation with evidence from individual studies. However, Annals has witnessed an increase in the frequency of “complex systematic overviews,” a term that we use to describe systematic reviews that use previous systematic reviews to construct an evidence base. Although the incorporation of past reviews into more recent syntheses can improve efficiency, complex systematic overviews present challenges. We describe those challenges and provide suggestions for overcoming them.

Why have complex systematic overviews become more common? First, as the number of published systematic reviews grows, so does the chance that those who set out to systematically review a topic will find previous reviews on the same or closely related topics. In addition, practice guidelines often require answers to not only one but several linked questions. A common example concerns screening interventions for which direct clinical trials that randomly assigned participants to a screening or control group are lacking. In such cases, guideline developers typically seek answers to a sequence of interrelated questions: How prevalent is the condition? What is its natural history? Is a good screening test available? What are the benefits (and harms) of early detection and treatment? When systematic overviews address interlocking questions, it is valuable for reviewers to tie the questions together in an analytic framework; this has become the standard practice for full evidence reviews commissioned by …