Selecting and Appraising Studies for a Systematic Review

Abstract

After thoroughly searching the potentially relevant literature for a systematic review, reviewers face the sequential tasks of selecting studies for inclusion and appraising these studies.Methodical, impartial, and reliable strategies are necessary for these two tasks because systematic reviews are retrospective exercises and are therefore prone to both bias and random error. To plan for study selection, reviewers begin with a focused clinical question and choose selection criteria that reflect this question. A detailed selection protocol that specifies the study designs and publication status of articles to be included is often helpful. Selection criteria are itemized on customized forms and are used to examine each potentially relevant primary study, usually by two different reviewers. In planning the critical appraisal of included studies, reviewers decide which clinical and methodologic study features require documentation. After choosing methods for evaluating study quality, reviewers construct customized appraisal forms and an explicit protocol for the actual evaluation. Some of the techniques commonly used to minimize the potential for error in study appraisal include duplicate, independent examination; blinding to study results and other identifying features of each article; and correspondence with study authors to clarify issues.

Ultimately, primary studies should be selected, appraised, and reported in sufficient detail to allow readers to judge the applicability of the review to clinical practice and to clarify the strength of the inferences that can be drawn from the review.