Physician groups frequently extol the sanctity of clinical peer review. In this issue of Annals, however, Hayward and colleagues [1] confirm that the way most physicians and review organizations do peer review to assess quality of care is unreliable [2]. Well-trained internists were asked to judge the appropriateness and quality of care on a general medical ward. Raters were taught a structured, implicit criterion approach, in which their perception of the relevant standard of care served as the basis for their judgment. Agreement among reviewers on the existence of most quality problems for single patients was generally poor. This study confirmed that peer review using implicit criteria is useful when judging groups of patients [3]; however, for single patients, it is useful only when care is egregiously bad and the relevant standard is obvious.

The findings would not be so important if the use of implicit criteria for judging the quality and appropriateness of care were not so widespread. Considerable money and energy have been spent in federal, state, and private peer review of the quality of care of individual patients, using implicit criteria and limited explicit criteria [4]. Such programs have created enormous hassle for conscientious doctors and countless heated rhetorical arguments with little evidence that they have substantially improved the quality of care [2-5].

As a member of a state disciplinary board from 1988 to 1992, I read hundreds of peer-review reports from the medical society committees mandated by state law to assess patients' potentially serious complaints. Few used predefined explicit criteria for what constituted a breach in standard of care. Many reports were "internally inconsistent" [6] as to whether care was substandard. Some concluded by stating that no standard was breached because an unfortunate outcome was possible rather than by deferring judgment until a representative sample of similar patients was examined to determine if this was an aberration or involved a pattern exceeding expected norms.

In contrast to the current study's finding, chart incompleteness did pose a problem. When charts contained little or no evidence to substantiate a diagnosis or treatment (not even the barest history or physical) or were unreadable even by doctors, they were often eliminated as not being evaluable and were considered evidence of record-keeping deficiencies rather than indications of substandard care.

Why, after almost a century of work by Codman [7, 8], Donabedian [9], Williamson [10, 11], Brook [12], Sanazaro [11], and others, are we still doing peer reviews so poorly? One reason is "the excellence deception" [13] whereby academic medical centers (which set the agenda for what is considered important) have been convinced of their own excellence. Consequently, the science of analyzing care and improving it has been given short shrift. This marginalization of quality-of-care evaluation as a respected topic in major teaching institutions is best illustrated by Codman's experience [8]. Now enshrined with Deming [14], another unheeded prophet, in the quality-improvement pantheon, Codman (a surgeon at the Massachusetts General Hospital) began in 1900 to record his cases and long-term follow-up. In 1914, when his proposal to institutionalize this "end results system" (now called "Outcomes Studies") was rejected by the hospital trustees he resigned in protest. As one of his supporters said, "There is nothing difficult about the system except the human nature part" [8].

From then until almost 1980, when general medicine divisions began to take the lead, the best research in quality of care was done by schools of public health and their health services research counterparts in the federal government and the private sector [15]. These groups have frequently been weakly linked and sometimes have even been at odds with schools of medicine and affiliated hospitals, where such research has too often been considered "soft science," if "science" at all. Unfortunately, the terms "hard" and "soft" have often been code words for "good" and "bad" or "elegant" versus "simplistic".

Such pejorative attitudes are unfair to the many disciplines that are so essential to medicine and to other fundamentally social endeavors. Epidemiology, informatics, logic, and ethics lie at the heart of quality-of-care research. Anthropology and sociology are also important because they strive to explain how the culture of the patient and of the patient care setting affects outcome [15, 16]. So is management sciences because the understanding of how institutions or systems work is essential to improving quality of care [17]. As with airplane crashes, most disastrous medical outcomes are not due to the action, inaction, or misjudgments of one person but result from multiple breakdowns [18, 19]. That is why reliance on the single-case approach, except for problem identification or for detection of physician outliers, is so misplaced.

Thus, the first prescription to improve peer review of the quality of care is to recognize and implement the techniques already developed by scientists outside traditional medical school departments [20]. At the same time, more clinician-scientists must be attracted into the field by giving health services research the same cachet as "bench research" [15, 21]. The American College of Physicians, among other groups, has taken a strong stand on the need for outcomes-oriented research and the development of practice guidelines [22, 23]. Most quality assurance of the future will not emphasize single-case review but rather the analysis of practice patterns by aggregating individual physician and group data across hospitals, high-volume centers of excellence, and private offices. The latter are the most invisible areas, especially shielding from view physicians without hospital privileges who often become isolated. How to measure quality in those settings, research pioneered by Peterson [24], Payne [25], and others, remains the most fertile area for investigation.

In the future, it is likely that reimbursement will be tied to systematic computerized collection of basic patient demographics, important diagnostic and prognostic information, and outcome data. Effectiveness of care will not simply be measured by mortality and serious morbidity but by functional status achievement compared with results that could be reasonably expected under the circumstances [19, 20, 26-28]. Process measures will include the use of drugs, tests, procedures, and preventive services, as well as documentation of necessary follow-up. Patient-satisfaction questionnaires will measure such things as appointment delays, waiting times, use of educational interventions, and even physician attitudes [29].

A second prescription for improving peer review involves increased education of residents and fellows in peer review using computerized tracking systems to help aggregate and analyze their own experience. This will prepare them for future practice and will give them a sense of how difficult it is to define a standard of care. No matter how difficult, however, we owe it to our patients to be able to define both the current attainable ceiling of excellence as well as the floor of minimally acceptable care. When trainees learn how little is known about appropriate norms and about how to adjust outcome data for severity of illness and other patient and institutional factors, they often become involved in the necessary research [17].

A third prescription involves establishing better systems for identifying and helping marginal doctors. First and foremost, medical schools and training programs need to overcome their reluctance to counsel students with serious character defects or insurmountable skill deficiencies into another line of work. Procedures to guarantee fairness will be essential to counter the not-so-veiled threats of lawsuits. Those physicians whose skills later become marginal due to laziness, greed, substance abuse, and desuetude of abilities or interest must be identified and educated where possible. Disciplinary boards are only useful in extreme circumstances. Their principal options are either dismissing complaints or taking punitive action (revoking or suspending a license). It is especially frustrating when board members review the care of "C^-, D" doctors who do not clearly and convincingly breach a standard. Previous use of a "community standard" to justify disparities in care has often led to clustering of such marginal physicians in certain locales. These physicians need help for their patients' sake, as well as for their own, but few remedial programs exist.

Continuing medical education accreditation is not sufficient because such doctors often do not seek the help they need. For this reason, some of the huge continuing medical education enterprise should be targeted at how marginal doctors practice where they practice. This will require direct observation by well-trained physicians [30], as well as the development of refresher miniresidencies. Such programs will be labor intensive and will expose the institutions and physicians to potential liability. Consequently, they must receive appropriate legal protection and remuneration. This will also involve insuring that the doctors who are doing peer review know how to do it properly and are also certified to be good doctors. Although there are many excellent and idealistic physicians who volunteer to serve on peer-review committees, these are not prerequisites for membership. In short, we must satisfactorily answer Juvenal's age-old question: "Quis custodiet ipsos custodes?" ("Who will watch the watchmen?") [31].

In the end, the patient's greatest guarantee of quality of care is the physician's character. External monitoring is useful in communicating expectations and helping us when we slip, but in the privacy of the examining room, it is us on whom the patient must rely. My experience on the disciplinary board convinced me that a number of bad apples still need to be removed from the medical barrel. However, Berwick [32, 33] is right to suggest that a total quality management (continuous quality improvement) approach be substituted for the "bad apples approach" to quality assurance. The best way for medicine to restore its eroding public confidence is by showing that quality comes first.