Persistent fears about editorial peer review [3, 4] include bias (for research attitudes, status, and sex), incompetence, timidity, plagiarism, suppression of facts, and delay in publication. Some evidence can be produced for all such charges. For example, the favoring by editors and reviewers of papers by top people from top institutions seemed to be confirmed by the much-cited study by Peters and Ceci [5]. They took previously published articles, disguised their origins, and then resubmitted these to the journals that had originally published them; largely unrecognized, these were then mostly rejected on scientific grounds. Nevertheless, the methods of this study have been challenged, although subsequent research has shown that removing such bias—masking the reviewer to the identities of authors—is not only feasible and effective but also produces better reviews than those from nonblinded reviewers [6]. It is now incumbent on all editors of serious journals, I believe, to at least be studying whether masking the identities of reviewers works in their circumstances.

The masking study was one of several important contributions to the First and Second International Congresses on Peer Review in Biomedical Publication, held in 1990 and 1993, respectively. Nevertheless, despite all these findings about process and outcome, we still lacked information about whether review improved manuscripts or not. Certain findings suggested that after the editorial process much accepted work is changed. Yankauer [7] found that as a result of peer review, 43 of 61 articles were substantially revised or totally rewritten before publication by the American Journal of Public Health. Lock [8] showed that 270 of 328 articles were changed (half of them substantially) as a result of reviewers' comments before publication by the British Medical Journal. Neither could say whether these changes were improvements; after all, authors, desperate to expand their curricula vitae, might consent to any change if this ensured publication. Nevertheless, for one important aspect, statistical handling, Gardner and Bond [9] did show in a before-and-after study that expert review objectively improved standards.

In their research, Goodman and coworkers [2] developed a questionnaire to assess the quality of manuscripts, which was then used by expert assessors unaware of the characteristics of the study. Most of the 34 designated features improved after peer review and editing, 4 of them in a statistically significant manner. Prominent among these were the discussion of study limitations, generalizations, and conclusions—all aspects, these authors point out, that readers especially rely on.

Self-evidently, these findings need the customary replication. In particular, I wonder how much of the reported improvement reflects peer review and how much the long-standing interest of Annals in "journalology" (the study of the structure, process, and outcome of journal editing). Thus, experienced staff editors might improve the rigor of a paper to an extent that would not happen elsewhere.

So the first question raised by these results is: Can any study be devised to separate out the individual contributions? Just possibly, although I wonder about an inescapable Hawthorne effect. Second, do these results imply that journals need more statisticians, methodologist reviewers, and expert staff editors? I hope not: Such articles should raise general consciousness, so that nonspecialist reviewers, editors—and especially authors—come automatically to appreciate what constitutes scientific rigor. Third, all such studies lead to the question: Are we developing an unrealistic view of editorial peer review, which is becoming a Holy Grail? Yes; all peer review can reasonably do is detect major defects of originality and scientific credibility, together with commenting on important omissions, the rigor of the arguments, and defects in the writing style. Peer review does not and cannot ensure perfection [10]; the gold standard for the quality of any reported work must remain time—whether it survives the customary 10 years or so before it is incorporated into review articles, textbooks, and other databases.

Publication has a biological quality to it. Witness the exponential increase in numbers of journals [11], the specialization of journals (so that currently at least five tiers exist [12]), and the recent slowdown in the appearance of new ones [13], as might be predicted in any biological system. Most such systems are wasteful, and publication is no exception. Consensus analysis [14] suggests that only 10% to 15% of all articles published on a particular topic are useful. Roughly half the articles published are never cited even once [15], and library use studies have shown that half the journals on the shelves are never looked at. Moreover, the general scientific adequacy of articles is low—a median of 1% to 6% in one study [16] of 2172 reports.

Fortunately, publication is a cheap method of disseminating information [17], and, crucially, it provides for the traditional Hegelian dialectic. Those important articles that passed the system have had a crucial role in scientific advance during the past 60 years. Because of some discouraging findings, no reason exists, therefore, to discontinue trying to improve peer review. In particular, I believe that editors could, and should, do more to help reviewers: Not only should they protect them against self-evident rubbish or general overload, but they must provide guidance about what help is needed and feedback on the eventual decision.

Editors also need continually to audit their procedures and apply the results of others to their own practices. For example, recent research has confirmed long-standing suspicions about the characteristics of good reviewers. A study of 226 reviews of 131 consecutively submitted manuscripts to the Journal of General Internal Medicine [18] showed that 86 (43%) of 201 reviewers produced good reviews (a grade of 4 or 5 on a 5-point scale). Logistic regression analysis showed that when a reviewer was younger than 40 years, from a top academic institution, well known to the editor commissioning the reviewer, and masked to the identity of the manuscript's authors, the probability that he or she would produce a good review was 87%. When a reviewer had none of these characteristics, the probability was 7%.

Nevertheless, prevention is better than cure, and peer review might be less strained if academia put its own house in order. For today, publication has often become hijacked only for showing academic activity, with an eye on research grants, tenure, promotion, and prestige. Although one study [19] showed that a top academic investigator could be expected to publish no more than eight articles every year, it also found that promotion went to those who had published double the number of articles of the nonpromoted—a recent league table [20] showed 11 physicians out of the world's 20 most prolific scientists, with 4 publishing every 5 to 10 days of the year during the previous decade. Add to this reputable practice an apparent recent increase in repetitive and salami publication, gift authorship, conflict of interest, and what has been described as an "arms race" for rapid publication [21] and the load on the editorial process becomes intense, and reviewers cannot be expected to give their best effort. Theoretically, the solution is simple: Return to a system where quality rather than quantity of new publications is the touchstone of excellence. Practically, also, the solution is simple: Limit the number of articles allowed on a curriculum vitae, as, after energetic proselytizing by Marcia Angell [22], was taken up in the Harvard Guidelines [23] on the conduct of research. Yet, academia seems to have been slow to adopt this more widely: Few of the institutions Edward Huth approached in the late 1980s had any such policies, and several responded brusquely to any suggestion that they should (Personal communication.).

Such a proposal begs answers to several questions. Would restrictions on the number of citable publications decrease the load on the editorial system? Would a decreased load then enable peer reviewers to give more time to fewer papers? And would more time spent on the later produce overall improvement? We have to put the concept to the test, yet intuitively, the answers would seem to be yes. For, although the system may be imperfect, nobody has produced a satisfactory alternative to editorial peer review. The aim of everybody concerned must surely be to make it better.