However, a process-outcome link between early readmission and the quality of care during the previous hospitalization has not been well established. Only 10 primary data studies [3-12] have examined the process of inpatient care directly and compared early readmission rates of patients whose hospital care was substandard with patients whose care was normative; results from these studies differ. For example, we [3, 4] previously found, as did Reed and colleagues [5], that patients who had changes in their medication regimen just before discharge were more likely to be readmitted within a month or less. However, in a case–control study of 292 patients from 50 Veterans Affairs hospitals, Ludke and colleagues [6] detected no differences in the adequacy of discharge planning or in medical stability at discharge between patients who were and were not readmitted within 14 days. A single-hospital retrospective cohort study by Hayward and colleagues [7] showed that quality-of-care ratings of the index stay did not differ between patients who were and those who were not readmitted within 28 days.

Early readmission is widely used as a quality-of-care indicator, although associations between it and the antecedent care process are not well established. We sought to determine whether the quality of inpatient care was associated with unplanned readmission within 14 days.

Methods

Top Methods Results Discussion Author & Article Info References

Participants

Patients enrolled in our case–control study were men discharged from a hospital stay for treatment of diabetes, chronic obstructive lung disease, or heart failure at a convenience sample of 12 participating Veterans Affairs hospitals in the southern United States between 1 October 1987 and 30 September 1989. Three hospitals were large, urban, referral centers affiliated with medical schools. Five hospitals were medium sized; all but 1 of these had a medical school affiliation. The remaining 4 hospitals were small and had no or limited affiliations.

Cases were men with an unplanned readmission to any Veterans Affairs hospital within 14 days of discharge from an index stay. Controls were men who did not have an unplanned readmission to a Veterans Affairs hospital within 14 days. The index stay was defined as the first hospitalization occurring during the 24-month period. The sampling frame was created using the computerized hospital discharge database of the Veterans Affairs medical system, the Patient Treatment File, which contains records of all hospital stays throughout the 159-hospital Veterans Affairs medical system. Three diagnosis-specific samples were created, with no patient appearing twice. Table 1 shows how samples were derived and lists reasons for ineligibility. Men discharged from an index stay with diabetes, chronic obstructive lung disease, or heart failure listed as the primary diagnosis were considered potentially eligible (International Classification of Disease codes available on request from the authors). All patients readmitted for any reason to any Veterans Affairs hospital within 14 days of the index discharge were placed on the list of potential eligible participants. Three patients who were not readmitted were randomly selected for each readmitted patient, and this 3:1 match provided a balance between the costs of chart review and statistical efficiency and power. The result of this sampling strategy was that patients who were not readmitted were group-matched to readmitted patients using hospital, diagnosis, and fixed 6-month period of discharge.

Review of the Process of Care

The process of care during the index stay was reviewed using a set of explicit, unit-weighted (0 or 1) process-of-care criteria specific for each diagnosis. (Criteria sets and instructions for use are available from the authors.) The criteria development and scoring processes have been described previously [13]. The sets were developed by panels composed of expert physicians and covered all elements of essential technical care of the hospitalized patient. Items were divided into three categories: criteria for the admission workup (history, physical examination, and initial tests), criteria for evaluation and treatment during the stay, and criteria for readiness for discharge. Many of the criteria were formulated as "if, then" statements, and a given criterion could be applicable or not applicable. The number of applicable criteria is a proxy for need for care. An adherence score, expressed as the percentage of applicable criteria that were met, was computed for each of the three categories of criteria. Each segment of the hospital stay was scored individually because different segments might influence the probability of readmission in a different manner.

The same physician and physician assistant served as chart reviewers for the entire study. Inter-rater reliability was assessed for each criterion using the statistic [14]. To account for the problem of a low score despite a high percentage of agreement, we categorized the criteria based on whether there was high or low percentage of agreement, and we eliminated criteria for which the reviewers had an observed percentage of agreement lower than that expected by chance and a score of less than 0.20 that persisted after training. The final diabetes criteria set included 47 admission-workup criteria, 42 evaluation and treatment criteria, and 11 readiness-for-discharge criteria. Respective numbers of criteria were 44, 19, and 10 for heart failure and 54, 21, and 9 for obstructive lung disease.

The reviewers traveled as a team to each site. One served as the administrative reviewer, and the other served as the quality reviewer. After ascertaining that the patient met the enrollment criteria, the administrative reviewer extracted data on severity of illness at admission using the Acute Physiology, Age, and Chronic Health Evaluation II (APACHE II) scoring method [15]; on comorbidity count (a list of 11 conditions with standardized definitions); and, in readmitted patients, on whether the readmission was planned or unplanned. The chart was then given to the quality reviewer, who applied the process-of-care criteria. The quality reviewer was kept blinded to the readmission status of the patient. Blinding is important because if reviewers know that an adverse outcome has occurred, they may rate the antecedent process of care as substandard [16]. Data on race, marital status, and number of admissions in the previous 2 years were obtained from the Patient Treatment File.

Readmission

The occurrence of readmission was tabulated from the Patient Treatment File and was therefore 100% complete for readmissions to any of the 159 Veterans Affairs hospitals. Readmissions to non-Veterans Affairs hospitals were not present in the Patient Treatment File and therefore could not be tabulated.

Statistical Analysis

Analyses were done separately for each disease. Bivariate comparisons were made using the Student t-test for continuous variables and the chi-square test for categorical variables. Student t-tests were two-tailed, and statistical significance was determined by a P value of less than 0.05. Multiple logistic regression [17] was used to estimate the unique effect of adherence scores on the probability of readmission, after controlling for patient demographic, illness-severity, and need-for-care variables as indicated by the number of applicable criteria. Because the number of criteria in each category differed for each of the three diseases, the numbers of applicable items were converted to standard normal variables to make the scales comparable. Conditional logistic regression was done to account for the matching of hospital and time period inherent in the design. The results were the same as those achieved with unconditional logistic regression, which indicated that hospital and time period were not confounding variables and that it was desirable to dissolve the matching. Unconditional logistic regression has advantages in the assessment of model fit because some regression diagnostic statistics (for example, Hosmer-Lemeshow deciles of risk) are not available for conditional logistic analysis. We examined all models and confirmed that the continuous variables conformed to a linear gradient. We assessed model fit by the Hosmer-Lemeshow deciles of risk statistic, by the log-likelihood chi-square analysis, and by the c-index (which corresponds to the area under the receiver-operating characteristic curve). The data did not violate general guidelines for multivariable analysis, and all models had acceptable fit.

The attributable fraction [18] was calculated in order to estimate the proportion of readmissions attributable to substandard inpatient care. The attributable fraction is equal to: (prevalence [relative risk –1])/(prevalence[relative risk –1] + 1), where prevalence is the proportion of controls who have the risk factor (substandard care). To obtain the prevalence of substandard care, adherence scores (which had been treated as continuous variables in the original logistic regression models) were arbitrarily dichotomized at the 25th percentile. Scores below that cut-point were taken to indicate substandard care. (A cutpoint between adequate and substandard care could not be empirically derived because the relation between adherence score and risk for readmission was linear.) We then re-ran the logistic models with this dichotomous score and used its odds ratio (OR) to approximate the relative risk. The computed attributable fraction is therefore adjusted for other explanatory variables. All analyses were done using the Statistical Analysis System (SAS Institute, Cary, North Carolina) [19].

Results

Top Methods Results Discussion Author & Article Info References

Bivariate Comparisons

Results of the bivariate comparisons are shown in Table 2. Patients with obstructive lung disease who had an unplanned readmission within 14 days were older than those who were not readmitted (67.5 compared with 65.5 years; P = 0.003). No other statistically significant differences in age, race, or marital status were found between patients who were and those who were not readmitted for any of the three diagnoses.

The number of applicable criteria can be viewed as a proxy for need for care. In all three conditions, patients with early readmissions had more numerous applicable admission-workup criteria (for diabetes, 33.5 compared with 32.2 criteria, P = 0.014; for obstructive lung disease, 52.9 compared with 52.4 criteria, P = 0.0001; for heart failure, 43.3 compared with 42.9 criteria, P = 0.033). Readmitted patients with obstructive lung disease also had more applicable evaluation and treatment criteria than those who were not readmitted (8.2 compared with 7.6 criteria, P = 0.003). Among patients with diabetes or heart failure, the number of applicable criteria did not differ between patients who were and those who were not readmitted. No statistically significant differences were noted between patients who were and were not readmitted in the number of applicable readiness-for-discharge criteria for any of the three conditions.

Statistically significant differences in one of the three categories of adherence scores were found for each of the three conditions studied. In patients with diabetes or heart failure, readiness-for-discharge scores were significantly lower in those who were readmitted than in those who were not readmitted. In patients with obstructive lung disease, admission-workup scores were significantly lower in those who were readmitted; readiness-for-discharge scores showed no statistical differences. Among patients with diabetes, the average readiness-for-discharge score in those who were readmitted was 7.2% less than that in patients who were not readmitted (67.3% compared with 74.5%, P = 0.0096). In patients with heart failure, the average readiness-for-discharge score was 4.4% less in those who were readmitted (75.0% compared with 79.4%, P = 0.012). In patients with obstructive lung disease, the average admission-workup score was 2% less in those who were readmitted (66.8% compared with 68.8%, P = 0.0187). Evaluation and treatment scores did not differ between patients who were and those who were not readmitted for any of the three conditions. (Readiness-for-discharge criteria for diabetes and heart failure are given in the Appendix; the admission-workup criteria for obstructive lung disease are available from the authors.)

Multivariate Analysis

For all three conditions, logistic regression models showed adherence scores to be statistically significant explanatory variables for unplanned readmission within 14 days, after controlling for demographic variables, severity of illness, and the number of applicable process criteria. In patients with diabetes or heart failure, lower readiness-for-discharge adherence scores were associated with a higher risk for readmission, whereas in patients with obstructive lung disease, lower admission-workup scores were associated with readmission. The respective process-of-care scores remained significant independent predictors even if the Bonferroni correction was made for the testing of three hypotheses (three categories of adherence scores) in each diagnosis. The coefficient of the respective adherence score for diabetes, obstructive lung disease, and heart failure was significant (P = 0.001, 0.013, and 0.016, respectively); the Bonferroni correction of the significance test was 0.05/3 = 0.0167.

Table 3 shows how decreased adherence scores correlated with increased risk for readmission; the data were obtained using the coefficients and standard errors from the models. Decreases in the adherence score were adjusted for the number of criteria applicable to a given patient (for example, if 4 readiness-for-discharge criteria were applicable to a given patient, 1 unmet criterion would cause a 25% decrease in the adherence score). In patients with diabetes, a 10-point decrease in the readiness-for-discharge score (1 unmet criterion in a patient to whom 10 were applicable) was associated with a 25% greater risk for readmission. A 30-point decrease (3 unmet criteria) was associated with an almost twofold (96%) increase in the risk for readmission. At most, 10 readiness-for-discharge criteria could have been applicable to a patient with heart failure, and each criterion would equal 10%. A 10-point decrease in the readiness-for-discharge score (1 unmet criterion if all 10 were applicable) was associated with an 18% increase in the risk for readmission. Up to 54 admission-workup criteria were applicable to a patient with obstructive lung disease, each worth 2%. A 10-point decrease in the admission-workup score (5 unmet criteria if all 54 were applicable) was associated with a 23% increase in the risk for readmission.

The proportion of readmissions that resulted from substandard care was estimated using the attributable fraction. To compute this, we defined substandard care as an adherence score below the 25th percentile. For patients with diabetes, using the same variables in the regression model, substandard care (a readiness-for-discharge score below the 25th percentile) was associated with more than a twofold increase in the risk for readmission (odds ratio [OR], 2.24; 95% CI, 1.11 to 4.49). The fraction of early unplanned readmissions in patients with diabetes attributable to substandard care, taking all other covariates into account, was 14.9% (1 of 7 readmissions). For patients with heart failure, using the same variables in the regression model, substandard care (a readiness-for-discharge score below the 25th percentile) was associated with nearly a twofold increase in the risk for readmission (OR, 1.94; CI, 1.21 to 3.10). The fraction of early unplanned readmissions in patients with heart failure attributable to substandard care, taking all other covariates into account, was 18.5% (1 of 5 readmissions). For patients with obstructive lung disease, using the same variables in the regression model, substandard care (an admission-workup score below the 25th percentile) was associated with a one third increase in the risk for readmission (OR, 1.36; CI, 0.95 to 1.96; note that the interval includes 1.0). The fraction of early unplanned readmissions in patients with obstructive lung disease attributable to substandard care, taking all other covariates into account, was 8.0% (1 of 12 readmissions).

Discussion

Top Methods Results Discussion Author & Article Info References

We found that the process of care during a hospital stay was associated with the likelihood of unplanned readmission within 14 days in patients with diabetes, heart failure, or obstructive lung disease, after adjusting for demographic characteristics, severity of illness, and need for care. We examined process elements that are under the control of medical care providers and assessed quality of care, using explicit standards developed by practicing clinicians affiliated with a medical school. Our data show that under certain circumstances, early unplanned readmission is associated with the quality of care during the previous hospital stay. Roughly 1 in 7 unplanned early readmissions in patients with diabetes, 1 in 5 in patients with heart failure, and 1 in 12 in patients with obstructive lung disease were attributable to substandard inpatient care, after other explanatory variables were taken into account.

One unique aspect of our study was that the explicit criteria for process of care were categorized so that they corresponded to the temporal sequence of a hospital stay. This allowed us to determine the relative influence of the three segments of the inpatient care process—the admission workup, evaluation and treatment during the stay, and readiness for discharge—on the probability of readmission. This strategy showed disease-specific differences for the segment exerting the most influence on the probability of readmission and therefore pointed out where clinicians might be able to improve the process of care and decrease readmissions.

In two of the three conditions we studied—diabetes and heart failure—releasing the patient before readiness-for-discharge criteria were met was associated with readmission. A readiness-for-discharge score below the 25th percentile was associated with a twofold increase in the risk for readmission in these conditions (OR, 2.24 in diabetes; OR, 1.94 in heart failure), after accounting for other covariates. Our readiness-for-discharge criteria covered clinical stability, education of the patient and family, and follow-up medical care. Physicians and nurses may be able to reduce the probability of early unplanned readmission in patients with diabetes or heart failure by ensuring that these simple discharge criteria are met before patients are released. Our data indicate that the risk for readmission decreases substantially with every additional discharge criterion that is met because the discharge criteria are few in number. In patients with obstructive lung disease, the completeness of the admission workup influenced the likelihood of early readmission; however, scores on the readiness-for-discharge segment did not influence readmission. It may be that incomplete admission workups in patients with obstructive lung disease fail to disclose the reason for the exacerbation and that unaddressed and, therefore, unresolved problems lead to subsequent readmission.

Our study provides important information on patient attributes (exclusive of the care process) that influence the probability of readmission. No statistically significant differences in age, race, or marital status between patients who were and were not readmitted were consistent among the three conditions studied. More consistency was found in the severity-of-illness variables. Both comorbidity count in patients with diabetes and number of previous stays in patients with obstructive lung disease were statistically different between patients who were and were not readmitted (none of the severity markers was significant in patients with heart failure). The number of readmissions attributable to substandard care in our study indicates that most unplanned readmissions (6 of 7 readmissions for patients with diabetes, 4 of 5 readmissions for patients with heart failure, and 11 of 12 readmissions for patients with obstructive lung disease) result from patient factors or process-of-care factors (or both) other than those measured in this study. How conditions at home after discharge affect the probability of unplanned early readmission needs to be examined. Also, little is known about how the content and quality of ambulatory care after discharge influence the probability of early readmission.

In hospital occurrence screening programs, charts of patients with an adverse event are reviewed by a nonphysician, and those with failing screening criteria are referred to a physician for peer review. Occurrence screening programs are generally inefficient because the adverse event is ultimately found to be due to substandard care in fewer than 2% of instances [20]. Our data indicate that using early unplanned readmission to screen for quality of care in patients with diabetes, obstructive lung disease, and heart failure is more efficient than using other commonly selected occurrence screens. Our study also suggests that reviewers using explicit criteria to examine the antecedent process of care in readmitted patients with diabetes, obstructive lung disease, and heart failure need not examine the entire hospital stay but can streamline the review by targeting only the relevant part of the hospital stay.

Our study was designed to examine the association between the process of inpatient care and early readmission on the level of patient and physician. Our findings indicate that when the physician does not adhere to standards of care, the patient's chances for an unplanned readmission within 14 days increase. Readers should not interpret our results to mean that hospitals with higher adjusted readmission rates are delivering care of lower quality. We did not set out to test that hypothesis, and that inference is not supported by our study design. A study determining whether hospitals whose readmission rates are outliers deliver care of systematically better or poorer quality would need a different design and analysis plan than we used. Such a study needs to be done, but many refinements are needed before early readmission rates calculated from administrative databases can be used to compare hospital quality.

We have previously suggested ways to improve methods for comparing hospital quality [21]. Our present study shows three important problems that must be overcome. First, we used a hospital discharge database to create lists of patients by diagnosis; however, a review of the hospital charts showed that in about one fourth of patients, the principal diagnosis was merely a coexisting condition and not the primary reason for admission. Targeting patients only by the diagnosis given in the discharge database would almost certainly affect the strength of the association between process and outcome. The second problem was our inability to distinguish unplanned from planned readmissions in our database. The association between quality of care and planned readmission probably differs from the association between quality of care and unplanned readmission. Third, many cite the lack of clinical severity-of-illness variables as an impediment to using readmission rates as hospital-quality indicators. In our study, however, the APACHE score—the only severity variable requiring manual extraction from the chart—was not a statistically significant predictor of readmission; comorbidity count and number of previous stays were significant predictors. These variables are readily available from most databases, although increasing evidence indicates that an accurate count of comorbid conditions may be difficult to obtain from present databases.

Our study was limited in its tracking of readmissions, a problem that has also occurred with other studies of readmission. We captured readmissions to any of the 159 Veterans Affairs hospitals, but we could not tabulate readmissions to non-Veterans Affairs hospitals. Most studies have tabulated only readmissions to the same hospital in which the index stay occurred, but such an approach seriously underestimates readmission rates because in the private sector, 22% of readmissions within 30 days are to different hospitals [22]. About 12% of readmissions within 6 months of an index stay in a Veterans Affairs hospital are to non-Veterans Affairs hospitals (data on early readmissions are unavailable) [23]. We would have misclassified patients readmitted to non-Veterans Affairs hospitals as controls rather than cases. This means that we underestimated the strength of the association between process of care and readmission if patients who received substandard care were more likely to be readmitted to non-Veterans Affairs hospitals than were patients who received adequate care.

Our results should be generalized with caution. We studied male veterans using Veterans Affairs hospitals. Persons who use Veterans Affairs hospitals are more likely to be poor and without health insurance than are those who use private hospitals, and the overall health status of poor persons may be worse. These factors may affect hospital utilization, in general, and early unplanned readmission, in particular. The strength of the apparent association between the process of inpatient care and readmission will vary depending on how accurately process of care is assessed, on how carefully readmission is characterized, and on how completely patient covariates are accounted for. Further work is needed to determine whether links exist between the process of inpatient care and early readmission in other types of hospitals, in women, and in other conditions.

Early readmission is a common occurrence in patients hospitalized for diabetes, heart failure, or obstructive lung disease [1]. Our findings indicate that the quality of inpatient care exerts a substantial influence on the risk for readmission and that even small improvements in the quality of care (adherence to a few additional process criteria) might decrease the risk for readmission. Research is needed to determine whether the frequency of unplanned early readmission is reduced after the quality of inpatient care is improved.

Appendix: Readiness-for-Discharge Criteria

Diabetes Mellitus

Criteria for All Patients Admitted for Problems Related to Diabetes

1. Acceptable blood glucose control is established (fasting blood glucose 3.3 to 13.9 mmol/L).

2. No changes in insulin or sulfonylurea doses have been made for at least 24 hours.

3. The chart indicates that the patient or family is able to do fingerstick glucose measurements. (This criterion was not used in the readmission study because inter-rater reliability was poor.)

4. The chart indicates that the patient or family understands the medication regimen and is able to administer insulin or oral sulfonylureas.

5. The chart indicates that the patient or family has been instructed in and understands the dietary regimen. (This criterion was not used in the readmission study because inter-rater reliability was poor.)

6. The chart indicates that the patient or family has been instructed in and understands foot care.

7. Plans for follow-up medical care are stated in the chart.

Criteria for Patients with Diabetes Who Are Admitted for Ketoacidosis or a Nonketotic Hyperosmolar State

1. The precipitating cause of loss of blood sugar control has been identified and corrected.

2. Other serious medical problems have been corrected or stabilized.

Criterion for Patients with Diabetes Who Are Admitted for Hypoglycemia

1. The precipitating cause of the episode has been identified and corrected.

Criteria for Patients with Diabetes Who Are Admitted for Lower Extremity Complications

1. The patient's temperature has been less than 37.8 °C for at least 24 hours.

2. The stump from amputation is healing and has no purulent exudate or cellulitis.

3. Rehabilitation has been planned or is under way in the new amputee.

Heart Failure

Criteria for All Patients with Heart Failure

1. Substantial improvement has occurred in symptoms (dyspnea, orthopnea) and signs (neck vein distention, S3 gallop, rales, edema, body weight).

2. Weight is stable or decreasing and is not increasing.

3. Temperature has been less than 37.8 °C for at least 24 hours.

4. Blood urea nitrogen and serum creatinine levels are stable or decreasing and are not increasing.

5. No change has been made in cardiac medications for at least 24 hours.

6. The serum digoxin level is less than 2.6 nmol/L and is not increasing.

7. The prothrombin time is stable and is not increasing.

8. The chart indicates that the patient or family, or both, understand the medication regimen.

9. The chart indicates that the patient or family, or both, understand the dietary regimen.

10. Plans for follow-up medical care are stated in the chart.