The Medicare Hospital Information Project is the first project to use HCFA's new philosophy for quality improvement. This project is designed to determine whether the Peer Review Organization and the provider communities can collaborate effectively on quality improvement initiatives through analysis of Medicare claims data and, with the commitment of limited additional resources, through medical record review. The Medicare Hospital Information Project was piloted by Peer Review Organizations from Connecticut and Wisconsin between May 1992 and March 1993. We describe the experience of the Peer Review Organization from Connecticut in the pilot phase of the project and highlight some of the scientific challenges confronting Peer Review Organizations in implementation of the Health Care Quality Improvement Program [2].

Our pilot project specifically involves analysis of the Medicare mortality data issued by HCFA from 1989 to 1991 [4]. Since their initial release, these reports have been a source of substantial controversy. The reports are derived primarily from claims submitted by hospitals to HCFA for payment of services rendered to Medicare beneficiaries and from mortality data from the Social Security Administration. Hospital representatives and the health services research community have criticized the reports as inaccurate, inadequately adjusted for risk [5], and easily misunderstood by the lay public [6]. Nevertheless, the reports contain potentially useful data for Peer Review Organization and hospital-sponsored quality improvement efforts.

In Connecticut, a three-member team consisting of the Peer Review Organization's physician clinical coordinator, statistician, and project coordinator spearheaded the pilot activities of the Medicare Hospital Information Project. This team worked closely with the Interhospital Study Committee, which incorporated physicians, nurses, hospital administrators, health services researchers, and medical records personnel from the Connecticut Hospital Association, the Connecticut State Medical Society, the Yale University and University of Connecticut Schools of Medicine and Public Health, the carrier fiscal intermediary (Travelers Insurance Company) for the Connecticut HCFA, and Connecticut hospitals.

After several preliminary meetings and review of the initial claims analysis done by the Peer Review Organization team, the Interhospital Study Committee suggested that a limited chart abstraction study be developed to evaluate potential sources of variation across hospitals in risk-adjusted mortality. Acute myocardial infarction was chosen as the focus condition because it is a high-volume, high-mortality diagnosis.

Methods

Top Methods Results Discussion Author & Article Info References

Use of the Medicare Mortality Data To Select the Study Hospitals

Six hospitals were selected for evaluation on the basis of observed/expected 30-day mortality rates (standardized mortality ratios) for acute myocardial infarction (principal diagnosis 410) during federal fiscal years 1989 to 1991 (1 October 1988 to 30 September 1991). The claims-based regression model for mortality used by HFCA—referred to as the Bailey-Makeham model [7]—forecasts the probability of death after admission to the hospital on the basis of several explanatory factors. These include patient age, sex, admission source (for example, elective or emergency), comorbid conditions (for example, cancer, cardiovascular disease, cerebrovascular disease, diabetes, liver disease, pulmonary disease, and renal disease), and previous hospitalizations (number and type, by risk level). The explanatory factors are combined in an exponential risk function. The model is constructed separately within numerous relatively homogenous diagnostic and procedure analytic categories, using all Medicare discharges nationwide. For each discharged patient, the model generates an estimate of the probability of survival to a specified number of days after admission. The estimates are then summarized at designated times (30 days, 90 days, and 180 days after admission). By aggregation over a given hospital's set of discharged patients (for all causes or within given diagnostic or procedure categories), these estimates yield average predicted mortality estimates. The predicted mortality estimates may then be compared with observed death rates to generate a set of standardized mortality ratios for that hospital. Using this method, we randomly selected two hospitals from each of the three terciles of the standardized mortality ratio distribution for all the patients admitted with acute myocardial infarction from 1989 to 1991. Throughout our report, we refer to these six hospitals as Low Mortality₁, Low Mortality₂, Average Mortality₁, Average Mortality₂, High Mortality₁, and High Mortality₂.

Development of the Expert Panel

An expert panel for acute myocardial infarction determined which elements to abstract from the charts in order to examine differences across the six hospitals in the following areas: 1) coding accuracy for the principal diagnosis of acute myocardial infarction; 2) severity of illness for acute myocardial infarction; and 3) quality of care for acute myocardial infarction assessed by several process measures. The expert panel for acute myocardial infarction convened with representatives from the study hospitals (although the hospitals were not informed that they were part of a special study) and from the Connecticut chapter of the American College of Cardiology. Final membership of the expert panel included the three members of the Peer Review Organization team, four cardiologists, three internists, and one nurse. On the basis of discussions among the panel members, a data collection instrument was then constructed, tested on a sample of charts available at the Peer Review Organization, and revised accordingly.

Patients

Fifty patients who were at least 65 years of age and had been reported on the hospital claims as having a primary diagnosis of acute myocardial infarction during federal fiscal years 1989 to 1991 were selected from the subset of admissions of Medicare beneficiaries at each of the six study hospitals that had been included in the mortality modeling project for HCFA. The hospitals participating in this demonstration project included teaching and nonteaching hospitals, community hospitals, and tertiary care centers.

Case Sampling

For each hospital, 50 consecutive patients with myocardial infarction were selected (patients with a principal diagnosis code of 410), starting with the discharges at the end of federal fiscal year 1991 (30 September 1991) and working backward in time until 50 patients were assembled. This sampling was done without regard to outcomes and without regard to whether the patients had catheterization or invasive surgery (coronary artery bypass graft surgery or percutaneous transluminal coronary angioplasty) during that admission or any later admission. For the smallest hospital among the six, it was necessary to go back 2 years to assemble a sample of 50 patients.

Data Collection Procedures

A copy of the medical record of each patient was requested from the hospitals; on receipt at the Connecticut Peer Review Organization, a single nurse-reviewer abstracted the data. This nurse-reviewer had 14 years of intensive care unit experience and 7 years of data abstraction experience. As questions arose during data abstraction, the reviewer met with the principal clinical coordinator to clarify and validate the data.

Confirmation of the principal diagnosis of acute myocardial infarction was based on the method of Iezzoni and colleagues [8]. The diagnosis was confirmed if two of the following three criteria were met: 1) typical symptoms [for example, chest pain, discomfort, pressure, or heaviness; arm, back, or jaw pain; nausea or vomiting; diaphoresis; sense of impending doom; dyspnea]; 2) electrocardiographic changes [for example, new Q waves; progressive ST-segment elevation or depression; progressive inversion of T waves]; and 3) abnormal levels of cardiac enzymes (for example, peak creatine kinase levels greater than twice the normal level; increased peak creatine kinase levels and an increase in the creatine kinase cardiac muscle fraction above 5%; peak lactate dehydrogenase [LDH] levels greater than 1.5 times the normal level and LDH₁ levels greater than LDH₂ levels). Symptoms were assessed from the admission history, progress notes, or cardiology consultation notes [if present]. Electrocardiographic evidence of myocardial infarction was abstracted from 1) the official interpretation of the electrocardiogram or the physician's interpretation as documented in the progress notes or 2) the hospital discharge summary. Abnormal levels of cardiac enzymes were determined using each hospital's reference range.

Killip Classification System

The medical record was also reviewed to determine the Killip class [9] as a measure of severity of acute myocardial infarction. The Killip classification system stratifies patients with acute myocardial infarction into four mutually exclusive levels according to the following criteria.

1. Class 1: no sign of congestive heart failure (no rales or crackles).

2. Class 2: rales (crackles) in one half or less of both lung fields.

3. Class 3: rales (crackles) in more than one half of both lung fields (pulmonary edema).

4. Class 4: cardiogenic shock (systolic blood pressure of 90 mm Hg or less; decreased urine output; cold, clammy skin; cyanosis; or mental obtundation).

Killip class was selected by the panel as a method for clinical risk adjustment because it represents a simple, previously validated method that is known to practicing clinicians. Patients were assigned to the highest-level Killip class for which there was documentation within the first 24 hours of admission.

Quality-of-Care Measures

To assess quality of care, we evaluated three process measures: 1) administration of thrombolytic therapy as part of the initial in-hospital management of acute myocardial infarction, 2) prescription of aspirin, and 3) prescription of ß-blockers at the time of hospital discharge as part of long-term therapy for patients with acute myocardial infarction. These process measures were chosen on the basis of the expert panel's knowledge of the medical literature and on local standards of care. The medical records were reviewed to identify contraindications to each of the three therapies (Table 1). For aspirin and ß-blockers, the therapy was considered indicated if the patient was alive at discharge and if no contraindications were identified in the medical record. To determine whether electrocardiographic indication criteria for thrombolytic therapy were present ( 1 mm of ST-segment elevation in two or more contiguous leads), a board-certified cardiologist (MJR) interpreted the first available electrocardiograms.

Missing Data

Demographic data and data used in the Bailey-Makeham regression model were available for all 300 patients. Killip class data were available for all these patients. Therapy use was coded as either present (if the medical record abstraction documented use of a therapy) or not present; thus, data or therapy use were available for all 300 patients.

Statistical Power

The total sample size of 300 patients was chosen to achieve approximately an 80% power to detect rate differences in therapy use between any one hospital and the mean of the other five hospitals that were 15% or more [ = 0.05]. The following assumptions were made: 1) Average utilization rates for therapy among eligible patients would be approximately 60% and 2) the average percentage of ineligible patients for a given therapy would be approximately 40%, with about 180 patients remaining for analysis. The 300 patients were selected from six different hospitals (with 50 patients each) to ensure a diversity of hospital settings in the study sample.

Statistical Analysis

To examine variation in the distribution of patients in each of the four Killip classifications across the six hospitals, chi-square analysis was used. Logistic regression models with 30-day mortality as the outcome measure were tested with Killip classification, predicted mortality rate values from HCFA, and various other patient-level and hospital indicator variables as the explanatory factors. The observed risk stratification performance of the Killip classification and HCFA's regression model was assessed with Kaplan-Meier product-limit survival analysis methods [10]. Risk stratification performance was specifically tested using the log-rank test of the statistical significance of differences in the observed interclass survival curves [11]. Agreement between the Killip classification and the model estimates from HCFA was assessed, after the HCFA data were collapsed to quartiles, by the statistic of Cohen [12].

To incorporate clinical data into the mortality risk adjustment, we combined Killip class with the 30-day predicted mortality from HCFA in a logistic regression model with 30-day observed mortality as the outcome measure. By using this model, 30-day standardized mortality ratios were calculated and compared with the original 30-day mortality ratios generated from the HCFA model. Relative hospital rankings by mortality ratio of the two methods were then compared.

Utilization rates for therapies were calculated using the number of patients with indications and without contraindications as the denominator for thrombolytic agents and the number of patients without contraindications as the denominator for aspirin and ß-blockers. Using Kaplan-Meier methods and restricting the denominator to patients with confirmed acute myocardial infarction who had indications for thrombolytic therapy without any contraindications to any of the three therapies, we compared 1-year survival rates for patients receiving at least one of the three therapies with survival rates of patients receiving none of the therapies. Risk-adjusted 1-year survival rates for these two patient groups were compared using Cox proportional-hazards model methods [11].

Results

Top Methods Results Discussion Author & Article Info References

Accuracy of Coding for Acute Myocardial Infarction

Of the 300 Medicare beneficiaries who were assigned a principal diagnosis code of acute myocardial infarction, 287 were confirmed as having acute myocardial infarction through application of the criteria developed by Iezzoni and colleagues [8]. Using these criteria, we found that the percentage of patients with confirmed acute myocardial infarction exceeded 90% at each of the six hospitals (range, 92% to 98%). Among the 13 patients who did not meet at least two of the criteria of Iezzoni and colleagues, 8 had typical symptoms only, 3 had electrocardiographic changes only, 1 had abnormal levels of cardiac enzymes only, and 1 did not meet any of the criteria of Iezzoni and colleagues.

Distribution of Killip Class

Overall, 78% of patients with confirmed acute myocardial infarction were in Killip class 1 or 2 (Figure 1). Although we could not exclude small differences across hospitals in Killip class with 50 patients per hospital, the distribution of Killip class did not vary among hospitals (chi-square = 21.12; P = 0.133). In particular, none of the six hospitals had a disproportionate number of either high-risk patients (Killip class 3 or 4) or low-risk patients (Killip class 1 or 2). Further, no monotonic increase was noted in Killip class from low-mortality to high-mortality hospitals. Of the two hospitals with more than 25% of patients in the high-risk Killip classes, one hospital was a lower-mortality outlier and one was a high-mortality outlier (Figure 1).

View larger version (44K): [in this window] [in a new window]   Figure 1. Distribution of Killip class among confirmed patients with acute myocardial infarction. The six Connecticut hospitals participated in the pilot study of the Medicare Hospital Information Project from 1988 to 1990.

Relation of Killip Class and Health Care Financing Administration Factors to Mortality

Both the Killip classifications Figure 2 and the mortality model data from HCFA Figure 3 were found to predict mortality over the 365-day survival follow-up period. As shown in Figure 2, the observed 1-year survival rates ranged from 25% for Killip class 4 to 82% for Killip class 1. Similarly, Figure 3 shows the Kaplan-Meier 365-day survival by quartiles of predicted mortality rates generated by the HCFA model. Observed 1-year survival varied from 52% for quartile 4 to 86% for quartile 1. In both these figures, the four survival curves depicted differ from each other (log-rank test, P < 0.01).

View larger version (12K): [in this window] [in a new window]   Figure 2. One-year survival from date of admission for confirmed patients with acute myocardial infarction among four Killip classifications. Six Connecticut hospitals participated in the pilot study of the Medicare Hospital Information Project from 1989 to 1991. See Methods section for Killip classifications (class 1, no congestive heart failure; class 4, cardiogenic shock).

View larger version (12K): [in this window] [in a new window]   Figure 3. One-year survival from date of admission for patients with confirmed acute myocardial infarction among quartiles of 180-day predicted mortality values from the Health Care Financing Administration. Six Connecticut hospitals participated in this pilot study of the Medicare Hospital Information Project from 1989. to 1991.

Effect of Killip Class on Risk Adjustment

We determined the influence of adding Killip class to the 30-day predicted mortality rates from HCFA in a logistic regression model with observed 30-day mortality as the outcome measure. Table 2 shows the distribution of 30-day standardized mortality ratios calculated by two models: 1) a model with a single explanatory factor—the HCFA model 30-day predicted mortality and 2) a model with two explanatory factors—Killip class and HCFA model 30-day predicted mortality. Logistic regression models were also evaluated, with in-hospital deaths, 180-day mortality, and 365-day mortality as the outcome measures. In all of these analyses, only the Killip class and HCFA model predicted mortality Table 2 were found to be statistically significant. Results for these analyses were similar to the results reported for 30-day observed mortality. Because of the consistency of these results and to simplify the presentation of these data, we report the interhospital comparisons of the standardized mortality ratios using the 30-day observed mortality and the two explanatory factors, Killip classification and HCFA predicted 30-day mortality. The wide confidence intervals around the standardized mortality ratios for each hospital reflect the limited number of patients selected in our pilot study.

As described in the Methods section, the six hospitals were sampled using their standardized mortality ratios for all admissions with the principal code of 410 for the period from 1 October 1988 to 30 September 1991. The standardized mortality ratios in Table 2 pertain to fewer than 50 patients per hospital (after exclusion of nonconfirmed patients), although the sampling of hospitals from terciles of standardized mortality ratio distribution was based on all admissions with a 410 principal diagnosis code over 3 years. Thus, the standardized mortality ratio—obtained using HCFA's risk-adjustment model—for the Average Mortality₁ hospital Table 2 is virtually identical to the values observed for the two low-mortality hospitals. After adjustment with the Killip class data obtained from the medical record, the rank ordering of the hospitals by standardized mortality ratio was not altered. Although definitive conclusions cannot be drawn because of the small number of patients and hospitals, five of the six hospitals had a standardized mortality ratio that tended to move toward the null value (1.00) with the addition of Killip class data.

Process Measures of Quality

Of the 287 patients with confirmed acute myocardial infarction, 150 (52%) had contraindications to thrombolytic therapy. Of the 137 patients with confirmed acute myocardial infarction who did not have contraindications to thrombolytic therapy, 68 (50%) had electrocardiographic indications for thrombolytic therapy. Overall, only 43% of patients with indications and without contraindications actually received thrombolytic therapy (Table 3). The percentage of patients appropriately receiving thrombolytic therapy did not statistically differ between low-mortality outlier hospitals (56% and 39%) and high-mortality outlier hospitals (18% and 45%) (chi-square = 1.40; P = 0.24).

For both aspirin and ß-blockers, the 225 patients who had a confirmed acute myocardial infarction and were alive at discharge were potentially eligible for treatment at discharge. In this group, 121 had contraindications for ß-blockers and 38 had contraindications for aspirin therapy, leaving as eligible for therapy at discharge a group of 104 patients for ß-blockers and 187 patients for aspirin. Overall, 41% of such patients received ß-blockers, whereas 73% received aspirin at discharge (Table 3).

The percentage of patients receiving either ß-blockers or aspirin was greater in low-mortality outlier hospitals than in high-mortality outlier hospitals (Table 3). For ß-blocker therapy at the two low-mortality outlier hospitals, 39% and 54% of patients discharged alive without contraindications were prescribed ß-blockers at discharge. Among the two high-mortality outlier hospitals, these percentages were 29% and 33%, respectively (P = 0.013). For aspirin therapy at discharge, aspirin was prescribed among patients discharged alive without contraindications at the following rates: 75% and 77% for the two low-mortality outlier hospitals; 41% and 66% for the two high-mortality outlier hospitals (P < 0.001).

Overall rates of use of at least one of the three therapies in eligible patients among the six hospitals are listed in Table 4. The two high-mortality hospitals had the lowest therapy utilization rates (35% and 51%) among the six hospitals. In comparison, the average utilization rate for the other four hospitals was 73%. Table 4 shows that the variation across hospitals in the use of these therapies was significant (chi-square = 8.66; P = 0.003).

Patient-Level Relation between Process Measures of Quality and Outcome

View larger version (10K): [in this window] [in a new window]   Figure 4. One-year survival from date of discharge for patients with confirmed acute myocardial infarction who did and did not receive therapy. Patients were discharged alive and were eligible for at least one of three therapies. One group received none of the three therapies (thrombolytic agents at admission, ß-blockers at discharge, or aspirin at discharge); the other group received at least one of the three therapies.

Discussion

Top Methods Results Discussion Author & Article Info References

Pilot data from the Connecticut Medicare Hospital Information Project show substantially better diagnostic accuracy for claims-based coding of acute myocardial infarction than has been reported previously. In a report of error rates in 1988 Medicare claims data from 30 California hospitals, the coding error rate for principal diagnosis (with all principal diagnoses considered) was found to be 9% [15]. Two studies [16, 17] using data from earlier periods (1974 and 1977) had indicated that diagnostic coding error rates were much higher, ranging from 18.5% to 42.8%. With regard to diagnostic coding errors that were specific to the principal diagnosis of myocardial infarction, Iezzoni and colleagues [8] found an average error rate of 26% among 15 hospitals according to data from fiscal year 1985 (October 1984 to September 1985), with error rates as high as 40% at these hospitals. In contrast, the 2% to 6% error rates found in our study for acute myocardial infarction in the six Connecticut hospitals from 1989 to 1991 are substantially lower than diagnostic coding error rates that were previously reported. Part of the explanation for this decrease can be traced to changes in coding guidelines that have been implemented since the 1988 study of Iezzoni and colleagues. In 1990, a fifth digit was added to the acute myocardial infarction code. This allows for differentiation between a patient admitted with an initial episode of care compared with a subsequent episode of care within 8 weeks of the acute myocardial infarction. The added precision allowed by this change would be expected to increase the percentage of patients meeting the clinical validation criteria of Iezzoni and colleagues. Thus, one would expect a higher accuracy rate in a study done after this coding change went into effect.

In a quality improvement project such as our study, exact utilization rates are less important than they are in research projects aimed at investigating process-outcome relations. Although we used broad contraindications for therapy in our study to give the benefit of the doubt to physicians not administering the therapy, appropriate use may still have been underestimated because of the limitations of chart documentation. Nevertheless, an important distinction of the Health Care Quality Improvement Program from previous Peer Review Organization activities is the focus on profiling medical care at the institutional level as opposed to the level of individual patients. We hope that providing such overview data stimulates institutions to collect their own internal data to validate and refine the external data. Our intent is to stimulate the use of data generated by the Connecticut Peer Review Organization in conjunction with internal interventions designed to improve care.

The Connecticut Medicare Hospital Information Proj-ect was viewed from its inception as a demonstration project and not as a project specifically designed to test the hypothesis that Medicare claims analyses using HCFA claims data-based risk adjustment represent the most efficient method of initiating hospital quality-of-care improvement efforts for Medicare beneficiaries. Specifically, although we did find that use of thrombolytic therapy correlated with variations in HCFA model-based mortality ratios across the six hospitals in Connecticut, our demonstration project was not meant to test the hypothesis that use of claims data as an initial quality improvement screening tool is preferable to some other approach (for example, hospitals and clinicians designing a quality improvement project for acute myocardial infarction without the benefit of Medicare claims data).

We believe that the results of the Medicare Hospital Information Project pilot study show that using Medicare claims data along with clinical data abstracted from medical records can provide useful information for quality management. One of the implications of these data appears to be that, if the projects of the Health Care Quality Improvement Program are to be maximally effective, data extracted from claims will probably have to be combined with clinically relevant data. Pattern analysis done with claims data may serve as a screening test in identifying institutions in which more resource-intensive, retrospective chart abstraction and primary, prospective data collection efforts will show opportunities for improvement.

A limitation of combining analysis of Medicare mortality data with focused chart abstraction is the delay that occurs in obtaining the Medicare mortality data. Our study was done from May 1992 to March 1993 with the most recent mortality data (fiscal year 1991) that were available from HCFA. One of the reasons for the follow-up study now in progress in Connecticut was that the data were perceived as already "old" at the time of feedback. This is especially important in a rapidly evolving area such as care of patients with acute myocardial infarction.

In the pilot project, data from the medical records were essential to identify specific opportunities for quality improvement. Although the Medicare mortality data have been criticized as an outcome measure of quality [5], we found these files useful in identifying hospitals in which further investigation might show important differences in process measures of quality. Specifically, one of the two high-mortality outlier hospitals was found to have the lowest rates of use of one or more of the three therapies, and both low-mortality outlier hospitals had the highest use of thrombolytic therapy. The medical record reviews were necessary to exclude other possible explanations for the observed variations in mortality ratios, such as misclassification of acute myocardial infarction in the Medicare claims files and inadequate risk adjustment because of the paucity of clinical information in the Medicare claims files. The medical record reviews were also required to identify specific process measures of quality care (for example, thrombolytic therapy use in candidates for thrombolytic therapy) where actual utilization rates differed from accepted practice patterns and thus represented opportunities for quality improvement.

Given the limited time frame and funding for our demonstration project, we could not generate evidence to determine whether the process of care for patients with acute myocardial infarction changed after the educational feedback phase of the Medicare Hospital Information Project. Before-and-after assessment of process-of-care measures would be required to examine the influence of educational feedback or some other aspect of Peer Review Organization activities on the actual process of care for Medicare beneficiaries.

Using the pilot experience of the Medicare Hospital Information Project in Connecticut, we conclude that Medicare claims data can be used by Peer Review Organizations to initiate cooperative projects to assess processes and outcomes of care and to identify areas in which quality improvements are possible. Further, additional clinical information not currently contained within Medicare claims files is necessary to resolve uncertainties about possible explanations in risk-adjusted variations in mortality. On-going condition-specific data collection efforts (such as those mandated by the Health Care Quality Improvement Program and the Medical Quality Indicator system and those efforts in progress through the HCFA's Cooperative Cardiovascular Project [1, 18]) have high potential to generate the level of clinical detail required to engage hospitals and health care personnel in the process of improving health care quality.

Although others have identified serious concerns with the Health Care Quality Improvement Program approach [3, 19], the Medicare Hospital Information Project pilot experience in Connecticut suggests that HCFA's methods to improve the quality of care for Medicare beneficiaries are feasible and potentially effective. Careful attention must be paid to the design of cooperative projects done by Peer Review Organizations under the Health Care Quality Improvement Program. The implementation of scientifically rigorous research designs will make it possible to measure and evaluate the actual effect of the efforts of peer review organizations on the process and outcomes of care provided to Medicare beneficiaries.