HCFA—Health Care Financing Administration

The frequency of coronary artery bypass graft surgery increased by 60% in the Medicare population between 1986 and 1988 [1]. The federal government estimated that 135 000 Medicare beneficiaries would have bypass surgery at more than 700 hospitals in 1991, at a total cost to Medicare of over $3 billion [1]. Assessing the quality of care provided to Medicare recipients is a priority of the Health Care Financing Administration (HCFA) [2].

Evaluation of the quality of care of hospital patients has recently focused on hospital-specific mortality rates [3-7]. This approach is facilitated by the availability of mortality statistics from administrative data sets and by the widespread acceptance of death as a readily measured and undesirable outcome of hospital care. However, the low frequency of postoperative death among surgical patients makes mortality an insensitive tool for evaluating surgery at many hospitals [6, 8]. Substandard patient care may result in problems that are common and serious but have little effect on 30-day mortality. Donabedian [9] and others [10] have suggested studying outcomes more closely related to specific conditions or procedures. This approach was recently used by Luft and colleagues [11] in a study in which nonfatal adverse outcomes of patient hospitalization were used as a measure of quality and by the HCFA in a recent report of adverse-outcome rates for selected procedures, including bypass surgery, in the Medicare population [2].

Another concern related to using hospital mortality or adverse-event rates to compare providers is the need to account for differences in illness severity at admission. When there is inadequate adjustment for severity of illness, conclusions about quality of care from mortality or adverse-event rates may be unjustified. In our study, we used chart-abstracted data to define and predict adverse events that occurred after bypass surgery in Medicare patients. The use of clinical information recorded at patient admission enables adjustment for severity of illness as well as the identification of the relation between illness severity and postoperative adverse events.

Methods

Top Methods Results Discussion Author & Article Info References

Study Population

Medicare patients 65 years of age or more composed our study sample; 2213 patients who underwent bypass surgery between January 1985 and June 1986 in seven states (Alabama, Arizona, Indiana, New York, Pennsylvania, Utah, and Wisconsin) were chosen from a larger study of hospitalized patients conducted by the HCFA in 1987. Peer review organizations in each of the seven states received identifiers for randomly chosen patients with specific ICD-9-CM diagnosis or procedure codes. These organizations directed the abstraction of clinical information according to the MedisGroups protocol [12-18]. MedisGroups is a proprietary software product developed by MediQual Systems, Inc. (Westborough, Massachusetts); MediQual staff trained the chart abstractors. Thomas and Ashcraft [16] recently found MedisGroups to have a high inter-rater reliability, comparable to that of APACHE II and diagnostic-related groups [16].

Information from Medicare, Part A, files (dates of death, hospital admission, and discharge; and ICD-9-CM diagnosis and procedure codes) was merged with clinical data to create the analytic files. Entrance into our study required the presence of one of the following ICD-9-CM codes for the bypass procedure: 36.10-36.16, 36.19, 36.20, and 36.30.

Definition of Adverse Events

Adverse events were conceptualized as serious postoperative complications potentially related to quality of care, resulting in a high likelihood of increased morbidity, subsequent intensive therapy, or prolonged hospital stay. Definitions of adverse events in the postoperative period were developed through review of the clinical literature on bypass surgery, consultation with medical and surgical subspecialists, and review of recommendations by expert panels convened by the HCFA. In addition to death within 30 days of hospital admission, 13 nonfatal adverse events occurring in the postoperative period were identified and defined as follows: new myocardial infarction by electrocardiogram; cardiorespiratory arrest for the first time; new congestive heart failure by chest radiograph; acute graft failure; new-onset thromboembolism; new-onset stroke; coma; mechanical ventilation for more than 48 hours; wound infection; bacteremia; acute renal failure as manifested by first-time peritoneal dialysis or hemodialysis or by a rise in the serum creatinine level from normal at admission to 442 µmol/L (5.0 mg/dL) or greater; transfusion of more than 6 units of whole blood or packed red blood cells; and an unplanned return to surgery.

Over 60 key clinical findings relating to severity of illness at admission were evaluated as predictors of the outcome ("any adverse event") (Appendix Table 1). To be studied, key clinical findings had to be of strong clinical interest or present in at least 10 patients at admission. Potential indicators of illness severity included elements of the medical history, such as a history of bypass surgery, diabetes, or stroke; laboratory abnormalities such as an abnormal serum potassium level; the presence of an infiltrate on the chest radiograph; and diagnostic test findings such as number of coronary arteries with stenosis of 50% or more. All key clinical findings were handled as dichotomous variables (that is, presence or absence of a clinical characteristic). Bypass surgery was defined as emergent when done on the day of admission. Acute myocardial infarction was considered present at admission when the electrocardiogram indicated acute myocardial infarction or the serum creatine kinase MB fraction was 0.04 or more.

Summary descriptive statistics were computed for all variables using the SAS software package (SAS, Cary, North Carolina). These included frequencies, proportions, and means plus standard deviations. Correlates of adverse events were identified using the t-test, chi-square test, and the Fisher exact test.

Regression models were developed in several steps to predict the occurrence of any adverse event, including death within 30 days of admission. In addition to selected key clinical findings considered to be indicators of illness severity, age (coded as a linear variable to the age of 85 years) and sex were included as explanatory variables in all models. The individual adverse events were combined into one single outcome variable, "any adverse event," because most did not occur frequently enough among the study patients to permit meaningful statistical modeling of single adverse events. Forward selection stepwise, ordinary least-squares regression was used for exploratory models; we have found that this type of analysis yields the same set of predictors as logistic regression and is more efficient. Variables were retained in regression models if their associated P values were 0.05.

The database was randomly split in half, and a model predicting any adverse event was developed for each half-sample. Each model was then fit to the other half-sample, allowing computation of a cross-validated R (²), which is calculated as 1 –SSE/SSM, where SSE is the sum of the squared differences between the outcome indicator and the probability of complication predicted by the regression equation, and SSM is the sum of the squared differences between the outcome indicator variable and the overall mean rate of adverse events [19]. A positive cross-validated R² was taken as evidence that the variables within the models had predictive power. Finally, all variables from either of the cross-validated models were used as candidates in a stepwise, logistic regression model procedure applied to the entire database. This process of model development was used to reduce the likelihood that explanatory variables would be selected through overfitting rather than systematic association.

A c-statistic was computed as a measure of the explanatory power of the logistic model. The c-statistic equals the probability that, for a randomly drawn pair of patients, one with and one without an adverse event, the model assigns a higher probability of having an adverse event to the patient who had one. The c-statistic is also equivalent to the area under a receiver-operating-characteristic (ROC) curve. A value of c near 0.5 indicates no discriminatory power, whereas a c of 1.0 indicates perfect discrimination between patients with and without adverse events [20].

To assess the model's goodness of fit, we calculated the Hosmer-Lemeshow statistic [21]; we also calculated expected numbers of patients with any adverse event, by decile of predicted risk.

Results

Top Methods Results Discussion Author & Article Info References

Demographic and clinical characteristics of the study sample are shown in Table 1. Eighteen percent of the patients were 75 years of age or more. Seventy percent of patients were men, and fewer than 3% of patients were nonwhite. The mean overall hospital stay was 15.0 ± 9.5 days. The rate of death within 30 days of admission was 6.6%. The HCFA recently reported a 5.7% rate of in-hospital death in Medicare recipients 65 years of age and more who had bypass surgery in the years 1986-87; the mean duration of hospitalization was 15.5 days [22].

Almost half of the study patients (44.4%) had a history of myocardial infarction, and 6.4% of patients had a history of bypass surgery. Emergent surgery was done in 5.7% of patients, and 9.9% had an acute myocardial infarction at admission. One or two grafts were placed in 19.2% of patients, and three or more were placed in 77.3% of patients; no information on the number of grafts was available for the remaining 3.5% of patients. The frequencies of all key clinical findings in the study sample and their distribution among patients with and without adverse events are shown in Appendix Table 1.

The prevalence of individual adverse events in the study sample is shown in Table 2; nearly one third of patients had at least one adverse event. Congestive heart failure and transfusion of more than 6 units of whole blood or packed red blood cells were each more common than death within 30 days of admission, occurring in 15.0% and 9.6% of patients, respectively. Every adverse event was associated with increased mortality. Cardiorespiratory arrest, coma, and acute renal failure were associated with particularly high mortality rates, ranging from 55% to 67%.

Patients experiencing adverse events differed in several ways from those who did not experience adverse events. Adverse events occurred in slightly older patients who had substantially longer hospitalizations (Table 3). Patients with adverse events had a mean hospital stay of 18.5 ± 13.2 days, whereas patients without adverse events had a mean hospital stay of 13.2 ± 6.2 days. The difference in duration of hospital stay was almost entirely attributable to longer postoperative stays. Among the 690 patients with adverse events other than death, 15.2% died within 30 days of admission compared with only 2.6% of patients without adverse events (P < 0.001).

Clinical findings at admission that related to comorbidity or preexisting severe coronary artery disease were also more common among patients with adverse events (see Table 3). These included a history of bypass surgery, diabetes, or myocardial infarction; emergent surgery; and acute myocardial infarction at admission (P < 0.05). Fewer patients experiencing adverse events had one- or two-vessel disease (22.9% compared with 27.2% of patients without adverse events, P < 0.05).

Table 4 shows the logistic regression model for predicting the occurrence of any adverse event; this model was developed using the technique described in the Methods section. The predictors of adverse-event occurrence are listed in descending order of the odds ratio magnitude. The single most powerful predictor of adverse-event occurrence was the history of bypass surgery (odds ratio, 2.78). Higher probabilities of experiencing an adverse event were also associated with emergent surgery; a history of chronic obstructive pulmonary disease; the presence of an infiltrate on the admission chest radiograph, a pulse of 110 or more; 10-year increments in age over 65 years; blood urea nitrogen of 10.7 mmol/L (30 mg/dL) or greater; acute myocardial infarction at admission; and a history of myocardial infarction. Patients with one- or two-vessel disease were less likely to experience adverse events.

The model c-statistic was 0.64. The Hosmer-Lemeshow statistic for the logistic model was 9.15, which was not statistically significant (for a chi-square with 8 degrees of freedom), indicating an acceptable fit of the model to the data. The ranking of cases by deciles of predicted risk (Table 5) shows that about 25% of patients in the four lowest risk deciles experienced adverse events compared with just over 50% of patients in the two highest risk deciles. The model ROC curve is presented in Figure 1.

View larger version (12K): [in this window] [in a new window]   Figure 1. Receiver-operating-characteristic curve for predicting any severe adverse event after coronary artery bypass graft surgery.

Discussion

Top Methods Results Discussion Author & Article Info References

We defined adverse events associated with bypass surgery using chart-abstracted clinical data; death within 30 days of admission was also included as an adverse event in our analysis. Altogether, one third of the study patients experienced one or more postoperative adverse events (including death), and nearly as many (31.2%) experienced at least one nonfatal adverse event. The frequency of adverse events was greater than that found in many other studies of bypass surgery, but the frequency of complications depends on how many adverse events are reported and how they are defined. Hammermeister and colleagues [23] reported a 15% rate of complications, excluding death, in over 10 000 veterans who had bypass surgery. Perioperative complications in this study that were also seen in ours included mechanical ventilation for more than 48 hours, myocardial infarction, stroke, renal failure, coma, and mediastinitis. Similarly, Hannan and colleagues [24], reporting on quality of care for patients undergoing open-heart procedures in New York State, noted a 14.1% rate of death, complications, or both. They reported complication rates for seven categories, with the highest rate being recorded for the "other complication" category. In both of these studies, the complications were based on reports from surgeons at participating institutions; no information about how complications were defined was reported. In another recent study of mortality prediction in 1300 patients undergoing open-heart surgery, Parsonnet and colleagues [25] found a nonfatal complication rate of over 50%. These investigators recorded "any and all postoperative problems". Finally, the HCFA used 24 ICD-9-CM codes to identify adverse events associated with bypass surgery in the 1986-1987 cohort of Medicare patients; the study found an adverse-event rate of 30.8%, which is closest to the rate we found [2].

The postoperative occurrence of adverse events in the study sample was clearly associated with greater severity of illness at admission, as seen in the univariate correlations of key clinical findings and adverse-event occurrence. Parsonnet and colleagues [25] also found an increasing frequency of postoperative complications among patients undergoing open-heart surgery who had greater illness severity at admission. In the multivariate analysis, we found that previous bypass was the single most powerful predictor of postoperative adverse-event occurrence (odds ratio, 2.8). Veterans with a history of open-heart surgery had an incidence of postoperative complications twice that of those who had no such history [23], and a recent study of predictors of in-hospital mortality after bypass surgery found that patients with a history of bypass surgery had 2.8 times the mortality of those undergoing the procedure for the first time [26].

The other elements of our logistic regression model predicting adverse events are also clinically plausible. In a different study population (U.S. veterans), Hammermeister and colleagues [23] also found that age, surgical priority, renal function, and a diagnosis of chronic obstructive pulmonary disease predicted nonfatal complications of bypass surgery. Other variables predictive in their multivariate analysis were New York Heart Association (NYHA) functional class; diagnoses of congestive heart failure, cerebrovascular disease, peripheral vascular disease, or diabetes; and the use of a preoperative intra-aortic balloon pump [23]. Interestingly, in the Veterans Affairs study, no variables representative of findings from coronary catheterization and angiography, such as extent and site of coronary artery stenoses, independently predicted adverse-event occurrence. Finally, a recent study of the morbidity associated with bypass surgery in patients at the Cleveland Clinic identified 13 clinical variables that were independent predictors of complications, including increasing age, an elevated serum creatinine level, chronic obstructive pulmonary disease, and previous and emergent bypass surgery [27]. The c-statistic for this morbidity model was 0.75, but the investigators noted that it was not successfully validated in a subsequent population of patients who also underwent bypass surgery at their institution.

Despite the strong association of illness severity at admission with postoperative adverse events, severity of illness, which, in our model, was based on clinical information at admission to the hospital, had only a modest ability to identify patients likely to experience postoperative adverse events, as indicated by the ROC analysis. Most of the variation in the occurrence of adverse events among our patients cannot be explained by these clinical data. There are several possible explanations for this. Some clinical information that we did not have, such as NYHA functional class, may predict adverse-event occurrence. In addition, adverse events may be strongly influenced by hospital- or physician-related factors, such as surgical volume, or other quality-of-care factors. Luft and colleagues [28] found, for example, that mortality rates in patients who underwent bypass were lower in hospitals with higher surgical volume [28]. Brook and colleagues [19], in a study that evaluated the prediction of complications associated with carotid endarterectomy in Medicare patients, also found that most of the variance in the complication and death rates after this procedure could not be explained by "clinically detailed illness variables".

Our study used a retrospectively assembled database and thus may not have had data on all variables related to illness severity and postoperative complications. For example, as can be seen in Appendix Table 1, we had no information about the number of coronary arteries with stenosis for 578 patients (26.1%). We suspect that these patients underwent coronary angiography during a previous hospitalization.

We based most of the definitions of postoperative adverse events on key clinical findings that MedisGroups terms "unusual occurrences". These key clinical findings are recorded whenever they occur during the hospitalization. The adverse events we defined from key clinical findings that are not deemed "unusual occurrences" by MedisGroups included acute renal failure, bacteremia, wound infection, and prolonged mechanical ventilation. The individual rates of these adverse events may actually be underestimated, because the key clinical findings on which their definitions are based are recorded only if they are found during the MedisGroups "discharge review" period, which spans only the last 3 days of hospitalization. Thus, a patient who has surgery on day 5 of hospitalization and is discharged on day 12 will have no MedisGroups record of abnormalities in renal function, bacteriologic culture results, or prolonged mechanical ventilation if such findings occur only within hospital days 6 through 9.

Finally, it is also possible that we missed some serious adverse events in patients who were discharged and then readmitted for postoperative complications, because we did not have information on readmissions. Even with these caveats, our findings are consistent with those of other studies, and it seems unlikely that additional clinical information could substantially improve the predictive power of our model.

The adverse events we studied had serious consequences for the patients who experienced them; an increased 30-day mortality rate was associated with the occurrence of each individual adverse event (see Table 2). Patients who experienced adverse events also had prolonged hospitalizations (see Table 3). Indeed, these findings lend some face validity to the idea that the occurrence of adverse events may indicate quality-of-care shortfalls [29], although further studies are needed to substantiate an association between postoperative adverse events and quality of care.

We have shown that postoperative adverse events are common in Medicare patients who undergo bypass surgery; the occurrence of such events is associated with greater severity of illness at admission, prolonged hospital stay, and increased mortality. However, severity of illness had only a modest ability to predict the occurrence of adverse events among our patients. These findings suggest that the occurrence of an adverse event may be a useful marker of suboptimal quality of care. First, the high frequency of these adverse events suggests that the problem of small numbers of events, as encountered in mortality rates [6], may be avoided. Second, a component of the unexplained variation in adverse-event occurrence may relate to variations in the care provided to patients who undergo bypass surgery. The concept that quality of care accounts for a substantial portion of the unexplained variation in outcome rates underlies recent work by the HCFA and other researchers [24, 26]. Further studies should explore whether adverse-event rates are reliable and valid screens for quality of care, as well as the extent to which adjustment for severity of illness influences expected adverse-event rates when providers of bypass surgery are compared.