 |
 |
|
Home |
Current Issue |
Past Issues |
In the Clinic |
ACP Journal Club |
CME |
Collections |
Audio/Video |
Mobile |
Subscribe |
Tools |
Help |
ACP Online
|
1 July 1994 | Volume 121 Issue 1 | Pages 11-21
Objective: To evaluate the effects of peer review and editing on manuscript quality.
Setting: Editorial offices of Annals of Internal Medicine.
Design: Masked before-after study.
Manuscripts: 111 consecutive original research manuscripts accepted for publication at Annals between March 1992 and March 1993.
Measurements: We used a manuscript quality assessment tool of 34 items to evaluate the quality of the research report, not the quality of the research itself. Each item was scored on a 1 to 5 scale. Forty-four expert assessors unaware of the design or aims of the study evaluated the manuscripts, with different persons evaluating the two versions of each manuscript (before and after the editorial process).
Results: 33 of the 34 items changed in the direction of improvement, with the largest improvements seen in the discussion of study limitations, generalizations, use of confidence intervals, and the tone of conclusions. Overall, the percentage of items scored three or more increased by an absolute 7.3% (95% CI, 3.3% to 11.3%) from a baseline of 75%. The average item score improved by 0.23 points (CI, 0.07 to 0.39) from a baseline mean of 3.5. Manuscripts rated in the bottom 50% showed two- to threefold larger improvements than those in the top 50%, after correction for regression to the mean.
Conclusions: Peer review and editing improve the quality of medical research reporting, particularly in those areas that readers rely on most heavily to decide on the importance and generalizability of the findings.
Author and Article Information
From Johns Hopkins University School of Medicine, Baltimore, Maryland; the University of Pennsylvania School of Medicine, Philadelphia, Pennsylvania; Harvard Medical School, Boston, Massachusetts.
ARTICLE
Manuscript Quality before and after Peer Review and Editing at Annals of Internal Medicine
Requests for Reprints: Steven Goodman, MD, PhD, Johns Hopkins University, Oncology Center, Division of Biostatistics, 550 North Broadway, Suite 1103, Baltimore, MD 21205.
Acknowledgments: The following "expert assessors" are thanked for their contributions to this project: Wendy Adams, MD; Eric Bass, MD; Greta Bunin, PhD; Jeffrey Carson, MD; Neal Dawson, MD; John Dickenson, MD; Kay Dickersin, PhD; Jose Escarce, MD; Harold Feldman, MD; Daniel Ford, MD, MPH; Martha Gerrity, MD; Matthew Gillman, MD, PhD; Deborah Grady, MD; Alan Hillman, MD; Thomas Imperiale, MD; Sharon Inouye, MD; Jeane Grisso, MD; Walter Kernan, MD; Thomas Keyserling, MD; Michael Klag, MD; Kurt Kroenke, MD; Seth Landefeld, MD; Suzanne Landis, MD; Valerie Lawrence, MD; Richard Lofgren, MD; Peter Margolis, MD, PhD; Wayne McCormick, MD; Alfredo Morabia, MD, PhD; Ann Naftinger, MD; Roberta Ness, MD; Warren Newton, MD; Sandra Norman, PhD; Patrick O'Connor, MD; Michelle Petri, MD, MPH; John Philbrick, MD; Jennifer Pinto-Martin, PhD; Peter Pompei, MD; Bruce Psaty, MD; Jeffrey Silber, MD, PhD; Rebecca Silliman, MD; John Steiner, MD; Ramon Velez, MD; Paul Whelton, MD, MSc; Larry Wissow, MD, MPH. The authors also thank Ms. Torii Ransome and Ms. Grace Lobb for their efforts in coordinating this project and Mr. Andrew Langman for assisting with the design of the questionnaire.
Related articles in Annals:
This article has been cited by other articles:
|
|
 |
|
  D. J. Benos, E. Bashari, J. M. Chaves, A. Gaggar, N. Kapoor, M. LaFrance, R. Mans, D. Mayhew, S. McGowan, A. Polter, et al. The ups and downs of peer review Advan Physiol Educ, June 1, 2007; 31(2): 145 - 152. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Sanderson, I. D Tatt, and J. P. Higgins Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography Int. J. Epidemiol., June 1, 2007; 36(3): 666 - 676. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 C. Laine, S. N. Goodman, M. E. Griswold, and H. C. Sox Reproducible Research: Moving toward Research the Public Can Really Trust Ann Intern Med, March 20, 2007; 146(6): 450 - 453. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. P. Landkroon, A. M. Euser, H. Veeken, W. Hart, and A. J. P. M. Overbeke Quality assessment of reviewers' reports using a simple instrument. Obstet. Gynecol., October 1, 2006; 108(4): 979 - 985. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. White A background to acupuncture and its use in chronic painful musculoskeletal conditions The Journal of the Royal Society for the Promotion of Health, September 1, 2006; 126(5): 219 - 227. [Abstract] [PDF]
|
|
|
|
 |
|
 M. E. Falagas and E. S. Rosmarakis Clinical decision-making based on findings presented in conference abstracts: is it safe for our patients? Eur. Heart J., September 1, 2006; 27(17): 2038 - 2039. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. S. Rosmarakis, E. S. Soteriades, P. I. Vergidis, S. K. Kasiakou, and M. E. Falagas From conference abstract to full paper: differences between data presented in conferences and journals FASEB J, May 1, 2005; 19(7): 673 - 680. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. N. Lohr Rating the strength of scientific evidence: relevance for quality improvement programs Int. J. Qual. Health Care, February 1, 2004; 16(1): 9 - 18. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. H. Edmunds Jr A gift from anonymous Ann. Thorac. Surg., January 1, 2004; 77(1): 10 - 11. [Full Text] [PDF]
|
|
|
|
|
|
C. Laine, C. Mulrow, and for the Editors Peer Review: Integral to Science and Indispensable to Annals Ann Intern Med, December 16, 2003; 139(12): 1038 - 1040. [Full Text] [PDF]
|
|
|
|
 |
|
 H. M. Druce, F. G. Hoppin Jr., and M. J. Tobin Peer Review and Manuscript Triage Am. J. Respir. Crit. Care Med., August 1, 2003; 168(3): 397 - 398. [Full Text] [PDF]
|
|
|
|
 |
|
 M. Moss, D. A. Wellman, and G. A. Cotsonis An Appraisal of Multivariable Logistic Models in the Pulmonary and Critical Care Literature Chest, March 1, 2003; 123(3): 923 - 928. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. White, G. Lewith, B. Berman, and S. Birch Reviews of acupuncture for chronic neck pain: pitfalls in conducting systematic reviews Rheumatology, November 1, 2002; 41(11): 1224 - 1231. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. J. Tobin Rigor of Peer Review and the Standing of a Journal Am. J. Respir. Crit. Care Med., October 15, 2002; 166(8): 1013 - 1014. [Full Text] [PDF]
|
|
|
|
|
|
F. G. Hoppin Jr. How I Review an Original Scientific Article Am. J. Respir. Crit. Care Med., October 15, 2002; 166(8): 1019 - 1023. [Full Text] [PDF]
|
|
|
|
|
|
E. J. Huth and K. Case Annals of Internal Medicine at Age 75: Reflections on the Past 25 Years Ann Intern Med, July 2, 2002; 137(1): 34 - 45. [Full Text] [PDF]
|
|
|
|
 |
|
 E. M. Balk, P. A. L. Bonis, H. Moskowitz, C. H. Schmid, J. P. A. Ioannidis, C. Wang, and J. Lau Correlation of Quality Measures With Estimates of Treatment Effect in Meta-analyses of Randomized Controlled Trials JAMA, June 12, 2002; 287(22): 2973 - 2982. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Jefferson, P. Alderson, E. Wager, and F. Davidoff Effects of Editorial Peer Review: A Systematic Review JAMA, June 5, 2002; 287(21): 2784 - 2786. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. Jefferson, E. Wager, and F. Davidoff Measuring the Quality of Editorial Peer Review JAMA, June 5, 2002; 287(21): 2786 - 2790. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. Wager and P. Middleton Effects of Technical Editing in Biomedical Journals: A Systematic Review JAMA, June 5, 2002; 287(21): 2821 - 2824. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 I. Sim, D. K. Owens, P. W. Lavori, and G. D. Rennels Electronic Trial Banks: A Complementary Method for Reporting Randomized Trials Med Decis Making, October 1, 2000; 20(4): 440 - 450. [Abstract] [PDF]
|
|
|
|
 |
|
 P. M. Rothwell and C. N. Martyn Reproducibility of peer review in clinical neuroscience: Is agreement between reviewers any greater than would be expected by chance alone? Brain, September 1, 2000; 123(9): 1964 - 1969. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. Juni, A. Witschi, R. Bloch, and M. Egger The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis JAMA, September 15, 1999; 282(11): 1054 - 1060. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. M. Pitkin, M. A. Branagan, and L. F. Burmeister Accuracy of Data in Abstracts of Published Research Articles JAMA, March 24, 1999; 281(12): 1110 - 1111. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. P. Purcell, S. L. Donovan, and F. Davidoff Changes to Manuscripts During the Editorial Process: Characterizing the Evolution of a Clinical Paper JAMA, July 15, 1998; 280(3): 227 - 228. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. A. Junker Adherence to Published Standards of Reporting: A Comparison of Placebo-Controlled Trials Published in English or German JAMA, July 15, 1998; 280(3): 247 - 249. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. M. Pitkin and M. A. Branagan Can the Accuracy of Abstracts Be Improved by Providing Specific Instructions?: A Randomized Controlled Trial JAMA, July 15, 1998; 280(3): 267 - 269. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. L. Hatch and S. N. Goodman Perceived Value of Providing Peer Reviewers With Abstracts and Preprints of Related Published and Unpublished Papers JAMA, July 15, 1998; 280(3): 273 - 274. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 T. Johnson Medicine and the Media N. Engl. J. Med., July 9, 1998; 339(2): 87 - 92. [Full Text] [PDF]
|
|
Article
