Scorecard Cardiovascular Medicine: Its Impact and Future Directions

Justification for Public Disclosure

Initially, most practitioners react defensively to the concept that medical outcomes should be measured and made available to physician peer review organizations, insurers, patients, and the public. Concerns are generally expressed about the ability of risk stratification to adequately adjust for differences and the ability of current methods to measure outcomes reliably. Despite these reservations, the process of scorecard medicine has been initiated and, in our opinion, will not be deterred. The arguments in favor of the “scorecard” approach from the clinical medicine viewpoint are that the process of measuring outcome provides a formal approach to improving individual decision making and system-wide allocation of resources, as shown in other complex work environments with the adoption of the Deming method of continuous quality improvement. The arguments in favor of disclosure are that patient payments and tax support through Medicare and Medicaid put large amounts of money into health care, the results of medical care can be measured, and those who pay are entitled to know what they are getting for their money.

Litigation has already paved the way for release of outcome data about individual physicians to the general public. Invoking the Freedom of Information Act, a newspaper (Newsday) sued the New York State Department of Health to obtain its database on bypass surgery and on cardiac surgeons. The Supreme Court of New York State ruled that the citizens were entitled to know these data in order to make informed decisions about their health care [11]. This “entitlement” has been popularized by the media with such video news magazine statements as, “You wouldn't buy a car with bad ratings, so why put yourself in the hands of a doctor with a questionable record?” and “It's one of the quirks of human nature that most of us spend more time shopping for a car or a VCR than choosing our doctors. Tonight our hidden cameras are going to show you what a difference comparison shopping could make” [12]. A newly published book guiding selection of the best top doctors and the top hospitals has quickly become a best seller [13]. The public clearly has the right and need to know more information about individual physicians' credentials, including outcome data, but the ideal and responsible way to provide this disclosure and the appropriate rules of disclosure remain uncertain.

Issues Raised by the Availability of Scorecard Medicine

As noted in Table 1, the release of adjusted outcome data presents several logistic issues that must be carefully considered and, in our opinion, have not been resolved [14, 15]. The most appropriate group of patients for constructing a “scorecard” must be decided on. The most widely publicized scorecards have been constructed for procedures (such as surgery or angioplasty), but in many cases a diagnostic category would be more appropriate (such as unstable angina) to prevent shifting of high-risk patients away from procedures. By operating or doing angioplasty on lower-risk patients and treating the high-risk patients medically, procedure-related mortality would decline, but mortality for the diagnosis may actually increase. For each patient group, whether identified by procedure or by diagnosis, the most appropriate risk-adjustment model must be developed. As our clinical and pathophysiologic understanding improves, these models must be updated. In our experience, most clinicians are uncomfortable with the methods of risk modeling and are skeptical about their accuracy. One notable example occurs when data important for risk adjustment are not available; most clinicians do not accept schemes of data imputation. Mortality may be a less important outcome than functional status, and, in some patients with terminal illness, earlier death may actually be desirable. Measurement of functional status outcomes is even more susceptible to clinician skepticism.

The statistical methods of outcome data analysis and reporting are evolving rapidly. Even among full-time clinical epidemiologic and statistical researchers, considerable controversy exists about the best modeling methods. The appropriate methods for the translation of large database analyses to individual practices and hospitals remain an area of uncertainty. The major issue in the statistical arena is closing the tremendous gap that has developed between researchers and clinicians and administrators who must make decisions based on the results of analyses. Making inferences from aggregated outcomes data without understanding the concepts of bias, confounding, and statistical power, in addition to other fundamental concepts of clinical epidemiology, could lead to erroneous inferences and policies.

Potential Impact

Preliminary evidence that the operator-specific scorecard approach can improve medical care recently came from New York State in a 3-year follow-up of the statewide, risk-adjusted mortality rate for coronary artery bypass surgery; this decreased from 4.3% to 2.7%, a 38% decrease [16, 17]. Interhospital shifting of high-risk patients to centers with the best record may, in part, have accounted for the improvement. For example, several cardiac surgeons refused to do high-risk bypass surgery on the father of a journalist in New York City soon after the dissemination of the Newsday report [18]. Eventually, after multiple consultations and refusals to accept the patient for surgery, this patient with poor left ventricular function had a successful bypass operation at a large-volume center. It is possible that surgeons might “game” (alter) the system by entering more high-risk data about their patients, thus improving the risk-adjusted mortality rate.

Physicians or hospitals with poor scorecards should examine clinical practice organization and skills and should change them; physicians or hospitals with exceptionally good outcomes should be rewarded and imitated. However, it is harder to interpret numeric differences within the range of acceptable results, especially when those differences are within the range of chance variation. In our opinion, internal dissemination within the profession about specific numeric outcomes within the acceptable range could motivate continuous examination and improvement, but considerable opportunity for misinterpretation of differences due to the play of chance exists with public disclosure of such differences. One of the most important concepts of modern clinical trials is the critical importance of adequate sample size to ensure that observed differences are certain. Displaying confidence intervals of the point estimates, as in Table 2, provides relevant information. However, an approach with a lower chance for misinterpretation would be to define an acceptable range and to publish specific point estimates for outcomes only when the observed value for a physician or institution lies outside of the acceptable range or when the differences between physicians or institutions are statistically significant.

Clearly, negative data can ruin a physician's reputation or career. The data for coronary angioplasty in New York State, which, like those for bypass surgery, were prospectively collected for all patients who had the procedure, led to the Department of Health's recommendation to discontinue a program in Elmira, New York [19], which had a 7% mortality rate compared with a 0.7% rate statewide. Moreover, the local publicity in this example was damaging to the credibility and professional reputation of individual physicians. On an institutional level, “hospitals with greater number of deaths than expected” Table 2[20] can have not only negative publicity but also fewer patients in the future, thereby potentially increasing the likelihood of future poor outcomes.

Directly or indirectly, medical scorecards may create a negative feedback loop so that hospitals or physicians with inadequate performance have decreasing clinical activity, with decreasing case loads, which further decreases the technical competence of the physician or team involved. Such forces will likely promote regionalization of procedures and a shift towards “centers of excellence”. A counterbalancing force, however, exists for the high-volume center with previous expertise that is referred more high-risk patients. The ladder to quaternary referral centers sets up a potential gradient of an abundance of high-risk patients clustered in a few hospitals. Conceivably, this could cause the development of strategies to prevent referral of high-risk patients to superior hospital sites and physicians.

For coronary artery bypass surgery, this potential avoidance of high risk epitomizes poor use of the procedure, because the patients with the highest risk are those who typically derive the greatest benefit from revascularization, such as those with left ventricular dysfunction, older age, and clinically significant comorbidity [21]. In order to avoid this scenario of an incentive to operate on the wrong patients, this issue must be effectively addressed with risk-adjusted models that do not penalize physicians for accepting patients at high risk. In fact, the logistic model can be fitted to reward physicians for operating on sick persons by adjusting the coefficients of the risk model to appropriate a slightly more than the expected number of deaths to the high-risk cohort.

Another possible outcome will be the delayed or inhibited development of new technologies, such as new percutaneous coronary revascularization procedures or unconventional but promising procedures using arterial conduits, such as the inferior epigastric or gastroepiploic arteries. Such procedures entail a learning curve during which errors can be made; the restrictive effect of having physicians under intense scrutiny will promote the use of conventional strategies. If physicians are restricted in their ability to explore advanced technology, many opportunities for technologic advancement may be lost.

A Proactive Proposal

We propose that clinicians aggressively confront the issues that make us veritable “sitting ducks” in the upcoming era of scorecard medicine. This approach involves routine measurement of outcome, assessment of the data for professional quality improvement, and disclosure to patients, at the hospital and physician level.

For example, typically hospitals doing inadequate case loads have suboptimal results with respect to mortality and coronary artery bypass surgery [22] and have an excessive requirement for emergency bypass surgery when doing coronary angioplasty [23]. Poor survival after cardiac transplantation is more closely related to institutional volume and coordinator experience [24]. Hospitals that have too few patients to gain competence in specific procedures (“fall below volume thresholds”) should either discontinue providing such services or should markedly improve their risk-adjusted outcomes (see Table 2). If this tack is not initiated by the hospital, a similar effect will likely be accomplished through lack of reimbursement by third-party payers. Such a rigorous approach could be pilot tested in a few states before being expanded to the country, as is currently occurring on a limited basis in New York, Pennsylvania, Iowa, and Colorado [12].

Too many cardiologists are doing coronary interventions and too many cardiac surgeons are operating. In the case of coronary angioplasty, 45% of members and fellows of the American College of Cardiology (> 9000 cardiologists) indicated in a recent survey that they do coronary angioplasty [25]. The average interventional cardiologist in the United States does fewer than 25 cases per year, despite the College's lenient guideline that each physician do at least 50 interventions per year [26, 27]. Similarly, the Joint American College of Cardiology and American Heart Association Task Force recommends that cardiac surgeons do at least 100 bypass operations per year [22], but in a review of the data now available in New York and Pennsylvania, more than one third of cardiac surgeons did not meet this criterion [11, 20] (Figure 1).

View larger version:

• In this window
• In a new window

• Download as PowerPoint Slide

Figure 1. Adopted from the Newsday, 9 December 1991 article, for all 140 cardiac surgeons doing coronary artery bypass surgery. Volume and mortality (mort.) are inversely related. Risk-adjusted hospital mortality data for cardiac surgeons from New York State.[9]

The problem of too many physicians is compounded by the lack of adequate training for many, who too frequently derive their “training” by attendance at a demonstration course. The recent commercial approval by the Food and Drug Administration of various new coronary interventions, including stents, rotary ablation, laser, and arthrectomy, exacerbates the critical mismatch of too many physicians and too little expertise [28]. Beyond training, outcomes, and proficiency, evidence exists of inadequate preprocedural evaluation in centers, particularly those without a postgraduate training program, doing coronary angioplasty [3].

As with hospitals, careful consideration should be given to the criteria for privileges of individual physicians. Low-volume physicians whose patients have poor outcomes should be prohibited from doing procedures. The minimum number of cases per year should be strictly enforced and consideration should be given to increasing the current standards, which are too permissive. Volume is not the only issue; prospectively, guidelines are necessary for the actions that should be triggered when indicators of poor-quality medicine are evident. Our preference is for these issues to be managed at the local level within the medical community. Indeed, availability of outcome data would likely alter the behavior of low-volume or poor-outcome practitioners. However, if these measures are unsuccessful, strategies ranging from admonitory communication to frank termination of procedural privileges could be used. The American Board of Internal Medicine, which initiated an Added Qualification Examination for Cardiac Electrophysiology, should upgrade the Interventional Cardiology Examination to improve the educational standards for future trainees. Novice practitioners should work with more skilled physicians and should restrict their patient selection to low-risk cases until they achieve adequate competence. Only certain physicians should develop expertise in certain new technologies rather than the dilutional effect of a small number of new technology cases for a large number of physicians. Further, training programs must decrease the number of specialists trained to do highly technical procedures, because a surplus of procedure-oriented specialists exists.

In the future, an ideal approach would be to shift “full disclosure” to inform patients. On the consent forms that patients sign before the procedure, the cardiologist and cardiac surgeon could include their actual risks and success rates, when appropriate, for the past several years, the cumulative number of procedures the physician has done, and the site's overall rate of complications for the procedure. This up-front disclosure to patients would be revolutionary because it is rarely practiced today; most physicians now provide an estimate of national statistics for consent form purposes. Actual instead of estimated rates for mortality and other complications, with 95% confidence intervals, could be certified by an independent peer-review group for authenticity. Such an approach represents a definite challenge, and the individual numbers would require updating on at least a semi-annual basis and require a reference group for comparison purposes, but this sort of information would help patients make decisions with confidence in their surgeon or physician. Many logistic issues about standard reporting formats and risk adjustments must be worked out for such “on-line” reporting to be feasible. Without furnishing such data up front, we are not truly providing informed consent.

We must do more clinical investigation. Research addressing improvements in diagnostic and therapeutic approaches to technology and pharmacology is vital to improve the quality of cardiovascular care. These efforts should focus on developing techniques that are easily transferable, less sophisticated, or operator-dependent and that substantially decrease the likelihood of a complication. Properly designed clinical trials evaluating new approaches to therapy should be categorized separately, particularly when high-risk patients are involved. Similarly, the revolution in information management that has made scorecard medicine possible also provides a means for physicians to band together to collect common databases. The formation of regional or national collaborative database groups could provide a common system to expedite clinical research and could give physicians major input and control over the types of data used to develop the scorecard.

The Future

In the 21st century, the practice of medicine will be vastly different from what we know today. At this juncture, we should anticipate a steady, stepwise, and major amplification of scrutiny of cardiovascular practitioners. What has already occurred in a few states will undoubtedly spread to all parts of the country in the next few years.

For too many years the medical profession has taken a reactive, resistant-to-change stance in the face of many legitimate and important societal problems; our credibility and moral authority have suffered as a result. If we can change from a reactive to a proactive stance in response to society's demand for greater accountability from the profession, we may prevent the potential ill will of the new era of scorecard cardiovascular medicine. Rather than polarizing the situation further, we need to work closely with the professional “scorekeepers” to develop rational regression models that adequately reflect our contemporary understanding of the underlying disease and expectations of the procedure and that reflect improved methods of collecting data that accurately categorize clinical risks and benefits. A tough stance needs to be adopted on low-frequency physicians and hospitals and to upgrade efforts at each hospital for a thorough, comprehensive quality assurance program. If we provide “full disclosure” at the hospital site and at the level of the individual physician, we can avoid dealing with hidden cameras from the media in the future and can maintain the highest standards of the profession.