Article    Table of Contents                    Full Text of this article Free    Figures/Tables List  Services    Send comment/rapid response letter    Notify a friend about this article    Alert me when this article is cited    Add to Personal Archive    Download to Citation Manager    ACP Search                            Get Permissions  Google Scholar    Search for Related Content  PubMed Articles in PubMed by Author:    Meade, M. O.    Richardson, W. S.    Related Articles in PubMed    PubMed Citation    PubMed

ACADEMIA AND CLINIC

Selecting and Appraising Studies for a Systematic Review

Maureen O. Meade, MD, FRCPC, MSc, and W. Scott Richardson, MD

1 October 1997 | Volume 127 Issue 7 | Pages 531-537

After thoroughly searching the potentially relevant literature for a systematic review, reviewers face the sequential tasks of selecting studies for inclusion and appraising these studies.Methodical, impartial, and reliable strategies are necessary for these two tasks because systematic reviews are retrospective exercises and are therefore prone to both bias and random error. To plan for study selection, reviewers begin with a focused clinical question and choose selection criteria that reflect this question. A detailed selection protocol that specifies the study designs and publication status of articles to be included is often helpful. Selection criteria are itemized on customized forms and are used to examine each potentially relevant primary study, usually by two different reviewers. In planning the critical appraisal of included studies, reviewers decide which clinical and methodologic study features require documentation. After choosing methods for evaluating study quality, reviewers construct customized appraisal forms and an explicit protocol for the actual evaluation. Some of the techniques commonly used to minimize the potential for error in study appraisal include duplicate, independent examination; blinding to study results and other identifying features of each article; and correspondence with study authors to clarify issues.

Ultimately, primary studies should be selected, appraised, and reported in sufficient detail to allow readers to judge the applicability of the review to clinical practice and to clarify the strength of the inferences that can be drawn from the review.

Author and Article Information

From the University of Toronto, Toronto, Ontario, Canada; and the University of Rochester School of Medicine and Dentistry, Rochester, New York.
Acknowledgment: The authors thank the clinical reviewer, Norman J. Wilder.
Requests for Reprints: Deborah Cook, MD, MSc, Department of Medicine, Division of Critical Care, St. Joseph's Hospital, 50 Charlton Avenue East, Hamilton, Ontario L8N 4A6, Canada.
Current Author Addresses: Dr. Meade: The Wellesley-Central Hospital, Room 244, Jones Building, 150 Wellesley Street East, Toronto, Ontario M4Y 1J3, Canada.

This article has been cited by other articles:

				B. Taylor, E. Wylie, M. Dempster, and M. Donnelly Systematically Retrieving Research: A Case Study Evaluating Seven Databases Research on Social Work Practice, November 1, 2007; 17(6): 697 - 706. [Abstract] [PDF]

				S. Braye and M. Preston-Shoot On Systematic Reviews in Social Work: Observations from Teaching, Learning and Assessment of Law in Social Work Education Br. J. Soc. Work, February 1, 2007; 37(2): 313 - 334. [Abstract] [Full Text] [PDF]

				B. J. Taylor, M. Dempster, and M. Donnelly Grading Gems: Appraising the Quality of Research for Social Work and Social Care Br. J. Soc. Work, February 1, 2007; 37(2): 335 - 354. [Abstract] [Full Text] [PDF]

				A. K. Bothe, J. H. Davidow, R. E. Bramlett, and R. J. Ingham Stuttering treatment research 1970-2005: I. Systematic review incorporating trial quality assessment of behavioral, cognitive, and related approaches. Am J Speech Lang Pathol, November 1, 2006; 15(4): 321 - 341. [Abstract] [Full Text] [PDF]

				T. T. Ng, M. L. McGory, C. Y. Ko, and M. A. Maggard Meta-analysis in Surgery: Methods and Limitations Arch Surg, November 1, 2006; 141(11): 1125 - 1130. [Abstract] [Full Text] [PDF]

				S Hoffmann and T Hartung Toward an evidence-based toxicology Human and Experimental Toxicology, September 1, 2006; 25(9): 497 - 513. [Abstract] [PDF]

				J. S. Berger, M. C. Roncaglioni, F. Avanzini, I. Pangrazzi, G. Tognoni, and D. L. Brown Aspirin for the Primary Prevention of Cardiovascular Events in Women and Men: A Sex-Specific Meta-analysis of Randomized Controlled Trials JAMA, January 18, 2006; 295(3): 306 - 313. [Abstract] [Full Text] [PDF]

				J J Swigris, W G Kuschner, S S Jacobs, S R Wilson, and M K Gould Health-related quality of life in patients with idiopathic pulmonary fibrosis: a systematic review Thorax, July 1, 2005; 60(7): 588 - 594. [Abstract] [Full Text] [PDF]

				R. Andraws, J. S. Berger, and D. L. Brown Effects of Antibiotic Therapy on Outcomes of Patients With Coronary Artery Disease: A Meta-analysis of Randomized Controlled Trials JAMA, June 1, 2005; 293(21): 2641 - 2647. [Abstract] [Full Text] [PDF]

				H. A. Bennett, A. Einarson, A. Taddio, G. Koren, and T. R. Einarson Prevalence of Depression During Pregnancy: Systematic Review Obstet. Gynecol., April 1, 2004; 103(4): 698 - 709. [Abstract] [Full Text] [PDF]

				M. B. Patwardhan, D. C. McCrory, D. B. Matchar, G. P. Samsa, and O. T. Rutschmann Alzheimer Disease: Operating Characteristics of PET-- A Meta-Analysis Radiology, April 1, 2004; 231(1): 73 - 80. [Abstract] [Full Text] [PDF]

				E. J. Ladas, J. S. Jacobson, D. D. Kennedy, K. Teel, A. Fleischauer, and K. M. Kelly Antioxidants and Cancer Therapy: A Systematic Review J. Clin. Oncol., February 1, 2004; 22(3): 517 - 528. [Abstract] [Full Text] [PDF]

				P. C. Nathan, L. Sung, M. Crump, and J. Beyene Consolidation Therapy With Autologous Bone Marrow Transplantation in Adults With Acute Myeloid Leukemia: A Meta-analysis J Natl Cancer Inst, January 7, 2004; 96(1): 38 - 45. [Abstract] [Full Text] [PDF]

				M. K. Gould, W. G. Kuschner, C. E. Rydzak, C. C. Maclean, A. N. Demas, H. Shigemitsu, J. K. Chan, and D. K. Owens Test Performance of Positron Emission Tomography and Computed Tomography for Mediastinal Staging in Patients with Non-Small-Cell Lung Cancer: A Meta-Analysis Ann Intern Med, December 2, 2003; 139(11): 879 - 892. [Abstract] [Full Text] [PDF]

				M. D. Brown, J. Lau, R. D. Nelson, and J. A. Kline Turbidimetric D-Dimer Test in the Diagnosis of Pulmonary Embolism: A Metaanalysis Clin. Chem., November 1, 2003; 49(11): 1846 - 1853. [Abstract] [Full Text] [PDF]

				T. M. King, K. Alperovitz-Bichell, P. C. Rowe, and H. P. Lehmann Is Epinephrine Efficacious in the Treatment of Bronchiolitis? Arch Pediatr Adolesc Med, October 1, 2003; 157(10): 965 - 968. [Full Text] [PDF]

				S. Hawker, S. Payne, C. Kerr, M. Hardey, and J. Powell Appraising the Evidence: Reviewing Disparate Data Systematically Qual Health Res, November 1, 2002; 12(9): 1284 - 1299. [Abstract] [PDF]

				E. A. Halm, C. Lee, and M. R. Chassin Is Volume Related to Outcome in Health Care? A Systematic Review and Methodologic Critique of the Literature Ann Intern Med, September 17, 2002; 137(6): 511 - 520. [Abstract] [Full Text] [PDF]

				A. Cranney, P. Tugwell, G. Wells, and G. Guyatt Meta-Analyses of Therapies for Postmenopausal Osteoporosis * I. Systematic Reviews of Randomized Trials in Osteoporosis: Introduction and Methodology Endocr. Rev., August 1, 2002; 23(4): 496 - 507. [Abstract] [Full Text] [PDF]

				B. O. Mustafa, S. W. Rathbun, T. L. Whitsett, and G. E. Raskob Sensitivity and Specificity of Ultrasonography in the Diagnosis of Upper Extremity Deep Vein Thrombosis: A Systematic Review Arch Intern Med, February 25, 2002; 162(4): 401 - 404. [Abstract] [Full Text] [PDF]

				M. K. Gould, C. C. Maclean, W. G. Kuschner, C. E. Rydzak, and D. K. Owens Accuracy of Positron Emission Tomography for Diagnosis of Pulmonary Nodules and Mass Lesions: A Meta-analysis JAMA, February 21, 2001; 285(7): 914 - 924. [Abstract] [Full Text] [PDF]

				D. Massel and S. H. Little Risks and benefits of adding anti-platelet therapy to warfarin among patients with prosthetic heart valves: a meta-analysis J. Am. Coll. Cardiol., February 1, 2001; 37(2): 569 - 578. [Abstract] [Full Text] [PDF]

				D. A. Fiellin, M. C. Reid, and P. G. O'Connor Screening for Alcohol Problems in Primary Care: A Systematic Review Arch Intern Med, July 10, 2000; 160(13): 1977 - 1989. [Abstract] [Full Text] [PDF]

				S. W. Rathbun, G. E. Raskob, and T. L. Whitsett Sensitivity and Specificity of Helical Computed Tomography in the Diagnosis of Pulmonary Embolism: A Systematic Review Ann Intern Med, February 1, 2000; 132(3): 227 - 232. [Abstract] [Full Text] [PDF]

				A. J. Lan, J. M. Colford, and J. MD The Impact of Dosing Frequency on the Efficacy of 10-Day Penicillin or Amoxicillin Therapy for Streptococcal Tonsillopharyngitis: A Meta-analysis Pediatrics, February 1, 2000; 105(2): 19e - 19. [Abstract] [Full Text]

				T. M. Shaneyfelt, M. F. Mayo-Smith, and J. Rothwangl Are Guidelines Following Guidelines?: The Methodological Quality of Clinical Practice Guidelines in the Peer-Reviewed Medical Literature JAMA, May 26, 1999; 281(20): 1900 - 1905. [Abstract] [Full Text] [PDF]

				M. K. Gould, A. D. Dembitzer, R. L. Doyle, T. J. Hastie, and A. M. Garber Low-Molecular-Weight Heparins Compared with Unfractionated Heparin for Treatment of Acute Deep Venous Thrombosis: A Meta-Analysis of Randomized, Controlled Trials Ann Intern Med, May 18, 1999; 130(10): 800 - 809. [Abstract] [Full Text] [PDF]

				C. A. Junker Adherence to Published Standards of Reporting: A Comparison of Placebo-Controlled Trials Published in English or German JAMA, July 15, 1998; 280(3): 247 - 249. [Abstract] [Full Text] [PDF]