Methods

Top Methods Data Collection Results Discussion Author & Article Info References

Our participants were selected from the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT), a multicenter study of outcomes and decision making for seriously ill hospitalized adults. Phase I of the study described the process of decision making and developed models to predict outcomes. Phase II was an intervention trial to evaluate the effect of prognostic information and enhanced communication. A description of the study design has been published [15].

Patients enrolled in SUPPORT had one of nine illnesses: acute respiratory failure, multisystem organ failure with sepsis, cancer with multiorgan system failure, chronic obstructive pulmonary disease, congestive heart failure, chronic liver failure, nontraumatic coma, colon cancer, or lung cancer. These categories were chosen to identify a cohort of patients with an anticipated 6-month mortality rate of 50% [15]. Patients with these illnesses were eligible for study participation if they met defined severity criteria at hospital admission or at any time during their stay in an intensive care unit. Patients were excluded if they did not speak English; had the acquired immunodeficiency syndrome (AIDS), were pregnant, or had multiple trauma; died within 48 hours of hospitalization; or were scheduled for discharge within 72 hours of admission. Phase I data collection took place between June 1989 and June 1991 at the Beth Israel Hospital, Boston, Massachusetts; MetroHealth Medical Center, Cleveland, Ohio; Duke University Medical Center, Durham, North Carolina; Marshfield Clinic/St. Joseph's Hospital, Marshfield, Wisconsin; and the University of California, Los Angeles, Medical Center. Phase II of the study took place from January 1992 to January 1994 at the same institutions.

Data Collection

Top Methods Data Collection Results Discussion Author & Article Info References

Baseline data were abstracted from medical records of the patients and were obtained from interviews with patients and surrogates between days 2 and 6 after study entry. Follow-up data were collected 2 months later by telephone interview. The surrogate was defined as the person who would make decisions about the patient's care if the patient was unable to do so.

Independent variables collected from the medical record included diagnostic category, other medical conditions, age, race, sex, education level, income, insurance status, enrolling hospital, and duration of hospitalization before study entry. Acute physiologic and neurologic status were determined using the SUPPORT physiology score [14] and a modified Glasgow coma score [16] measured on the third day after study entry. The patient and surrogate hospital interviews (Appendix 1) included questions on reported performance of activities of daily living 2 weeks before study admission, using a modified version of the Katz Index of activities of daily living [17]; exercise tolerance 2 weeks before admission, using a modification of the Duke Activity Status Index [18]; and quality of life at the time of interview using a 5-point rating scale [19]. During a telephone interview at month 2, the patient was asked about his or her functional status using the Sickness Impact Profile [20] and the patient and surrogate were asked about the patient's activities of daily living and overall quality of life. The Katz Index of activities of daily living [17] was modified so that it could be obtained during an interview with the patient. Walking was added to the six basic activities included in the original index. Questions were worded to obtain a report of actual performance rather than perceived capacity. This modified index was scored on a 7-point scale, with each point indicating dependence on assistance for one of the following basic functions: eating, continence, toileting, transferring, bathing, dressing, and walking. Patients and their surrogates were asked to report on functioning for 2 weeks before study entry in order to approximate a baseline level of functioning.

The Duke Activity Status Index is a patient-reported measure of ability to do several personal, household, and recreational activities, each of which was calibrated to its metabolic requirements to assess cardiovascular capacity [18]. In our study, the item on sexual function was deleted and three items about yard work, moderate recreation, and strenuous recreation were combined into a single item. Questions were asked in hierarchical order so that patients unable to do less strenuous activities were not asked about more demanding activities. Patients and their surrogates were asked about the patient's ability to do as many as 11 activities 2 weeks before study admission. The Duke Activity Status Index was scored so that a higher score indicated greater metabolic capacity. Quality of life at the time of the interview was assessed using a 5-point scale ranging from excellent to poor.

The Sickness Impact Profile is a generic health status measure that assesses sickness-related dysfunction [20]. It has 136 items grouped in 12 areas of activity: sleep and rest, emotional behavior, body care and movement (for example, bathing or transferring from bed to chair), eating, home management, mobility (for example, staying at home), social interaction, ambulation, alertness behavior, communication, recreation, and work. Sickness Impact Profile scores range from 0 to 100, with a higher score indicating greater dysfunction. The Sickness Impact Profile has been tested and used extensively in clinical and health services research studies [21, 22] and has shown consistently high reliability coefficients and excellent convergent and discriminant validity [23]. The clinical validity and responsiveness to change of the Sickness Impact Profile have also been shown [22].

Management of Missing Data for Independent Variables

Because interview data were not available for every patient and surrogate and because analyses done on subsets of a database may be biased (for example, persons not responding may be sicker than respondents), we developed a substitution and imputation strategy for missing independent variables. We gave priority to patient rather than surrogate responses about functioning and quality of life. When the patient response was missing, a calibrated surrogate response was used. On the basis of the subset of cases for which patient and surrogate responses were available, surrogate responses were calibrated to achieve a distribution similar to that of patient scores. When neither patient nor surrogate response was available, an imputation strategy was used. Ordinal logistic or linear regression models containing age, SUPPORT physiology score at day 3, diagnosis, number of additional diagnoses, cancer diagnosis, site, interview status, and length of time in the hospital before study entry were used to predict surrogate-reported activities of daily living, quality of life, and Duke Activity Status Index. Surrogate responses were estimated instead of patient responses because surrogate responses were available for a broader range of situations. The estimated surrogate responses were then calibrated to the patient distribution in the same way that actual surrogate responses were calibrated (Appendix 2).

Development of Predictive Model

To estimate the probability that survivors had severe functional limitation after 2 months, we used a previously established definition of severe functional limitation indicating problems that would require nearly constant personal assistance (that is, Sickness Impact Profile scores 30 or patient-reported activities of daily living scores 4). If the patient could not be interviewed, a surrogate report of an activities of daily living score of 5 or more was used because for cases in which patient and surrogate responded, a surrogate-reported score of 5 corresponded to a patient-reported score of 4. One hundred eleven patients were classified as severely limited on the basis of both Sickness Impact Profile scores and activities of daily living scores, 121 were classified as severely limited on the basis of Sickness Impact Profile scores alone, and 349 were classified as severely limited on the basis of activities of daily living scores alone. Patients who were comatose or intubated at month 2 were also classified as severely limited (n = 11).

Analysis

On the basis of published reports [8, 10, 24-28] and clinical experience, we hypothesized that disease type and patient demographic characteristics, severity of illness, previous functional status, and self-rated quality of life would be important determinants of future functional status. Because lead time may be important in prognostic models [3, 29], we included a variable for time spent in the hospital before study enrollment.

Candidate variables were entered into a backward stepwise logistic regression model. A variable was defined as important and was retained if its chi-square statistic was greater than twice its degrees of freedom [30]. Five potential interactions between prognostic variables were prespecified using a published report [31] and previous experience: age and baseline score for activities of daily living, age and number of days spent in the hospital before study admission, activities of daily living score and SUPPORT physiology score at day 3, activities of daily living score and disease group, and SUPPORT physiology score and disease category.

For categorical variables (quality of life, diagnosis, sex, and site), adjusted odds ratios and 95% CIs were derived from the logistic regression coefficients and their standard errors. For continuous variables (age, SUPPORT physiology score at day 3, activities of daily living score, Duke Activity Status Index, and hospital days before study entry), adjusted odds ratios and 95% CIs were estimated for specified increments (for example, from 60 to 70 years for age).

Assessing Model Performance

Predictive discrimination was assessed using the area under the receiver-operating characteristic curve [32, 33]. This statistic represents the concordance between predicted probabilities and observed outcomes for all possible pairs of patients with different outcome status. A receiver-operating characteristic area of 1.0 indicates perfect discrimination, whereas an area of 0.5 indicates that the model discriminates no better than chance. We plotted calibration curves to examine the model's fit across the range of predicted risk for severe functional limitations when compared with the observed outcome. A nonparametric smoother was used rather than the traditional approach of grouping the data in deciles because the groupings could be arbitrary [34]. We validated the model on the independent set of control patients in phase II of SUPPORT using receiver-operating characteristic area and calibration. All statistical computations were done using the S-Plus Statistical Language Version 3.1 (Statistical Science, Inc., Seattle, Washington) on UNIX and the SAS version 6.07 (SAS Institute, Cary, North Carolina).

Results

Top Methods Data Collection Results Discussion Author & Article Info References

Participants

A total of 4301 persons were enrolled in phase I of SUPPORT. Scheduled data-reliability surveys indicated differential interviewing technique for the 2-month interview at one site, and 682 patients with 358 interviews at this site were excluded from the analysis, leaving 3619 eligible patients (inclusion of these patients in the analyses did not alter the relative importance of predictor variables in the model). Of these, 1306 persons (36.1%) died before the 2-month interview, and 567 did not have data for the dependent variable (Sickness Impact Profile or activities of daily living score at 2 months) because of an inability to communicate and the lack of a surrogate interview (n = 307), because of refusal or no response (n = 151), or because data were missing for other reasons (n = 109); thus, 567 persons were excluded from the analysis. The final sample for model development comprised 1746 patients who survived for 2 months and from whom valid patient or surrogate outcome data were obtained (Figure 1). The independent validation set consisted of 2478 patients with functional status data at 2 months out of 4804 patients in phase II of SUPPORT (1614 died and 712 did not have functional status data at 2 months because of an inability to communicate and no surrogate [n = 122], because of refusal or no response (n = 373), or because data were missing for other reasons [n = 215]).

View larger version (13K): [in this window] [in a new window]   Figure 1. Derivation of study sample.

Response Rates

For patients who survived 2 months, the overall response rate for day 3 interviews was 89.5%. There were 195 imputed values for baseline scores of activities of daily living, 239 imputed values for baseline scores of quality of life, and 205 imputed values for baseline scores of the Duke Activity Status Index for participants in the study. A total of 257 (14.7%) patients had at least 1 value imputed. Compared with the 567 patients without data for the dependent variable at month 2, interviewed participants in the study depended on assistance for slightly fewer activities of daily living and had slightly better quality of life at baseline (Table 1; P 0.001).

Univariable Analysis

At 2-month follow-up, the number of patients reporting severe functional limitations had nearly tripled to 590 (34%). Among patients with no dependencies in activities of daily living 2 weeks before study entry, 21% had developed severe limitations 2 months later. This proportion increased directly with the patient's level of baseline dependency, as high as 70% among patients who reported 4 or more baseline dependencies. On the other hand, 30% of patients with 4 or more dependencies in activities of daily living at admission showed improvement by 2 months (Table 2). Activities of daily living scores and Duke Activity Status Index at baseline were moderately correlated with one another (Spearman r = 0.49; P < 0.0001), as were activities of daily living and Sickness Impact Profile scores after 2 months (Spearman r = 0.57; P < 0.0001). Functional outcomes varied for patients with different primary diagnoses. At 2 months, the proportion of patients with severe functional limitations ranged from 13% survivors with colon cancer to 71% of survivors with coma.

Among patients with severe functional limitations who rated their own quality of life at month 2, approximately 27% rated their quality of life as poor; 46% rated it as fair; 24% rated it as good; 10% rated it as very good; and 3% rated it as excellent. Patients with severe limitations at month 2 were less likely than patients without severe limitations to survive to 6 months (68% compared with 88%; log-rank test, P < 0.0001).

Multivariable Analysis

In the logistic regression model (Appendix 3), eight variables were independent predictors of severe functional limitations at 2-month follow-up. Table 3 shows the odds of severe limitations after 2 months for each significant variable in the model while other variables are held constant. For example, the odds ratio for severe limitations is 1.95 for a patient with one baseline dependency in activities of daily living compared with no dependencies. The strongest independent predictor of severe functional limitations was activities of daily living scores 2 weeks before study entry. Severe limitation at 2 months was also associated with poorer quality of life at baseline, with more limited exercise tolerance before admission, with worse physiologic imbalance or coma early during hospitalization, and with longer hospitalization before enrollment. Compared with patients who had acute respiratory failure, patients with congestive heart failure had a better prognosis for functioning at 2 months, whereas patients with coma had a worse prognosis. For patients younger than 60 years, age was not a risk factor for severe limitations. However, risk began to increase after the age of 60 years. None of the prespecified interactions was statistically significant.

Evaluation of the Prognostic Model

The area under the receiver-operating characteristic curve was 0.79 for the model, which indicates reasonably good discrimination. A model that only used baseline scores for activities of daily living to predict future limitations discriminated fairly well, with a receiver-operating characteristic area of 0.72. For 27% of patients, however, the estimated risk for poor functioning differed by more than 0.2 when compared with the risk determined using the complete model. In the validation sample, the area under the receiver-operating characteristic curve was 0.75 (0.70 for activities of daily living alone). Calibration curves showed reasonably good agreement between predicted and actual dysfunction across the range of predicted risk and actual outcome in the development and validation data sets (Figure 2).

View larger version (19K): [in this window] [in a new window]   Figure 2. Calibration curves. In both panels, the solid line denotes perfect calibration between observed frequency and predicted risk for severe functional limitations 2 months after hospitalization. The dotted line indicates actual calibration. Top. Data for 1746 seriously ill patients in the model development sample. Bottom. Data for 2478 seriously ill patients in the independent validation sample. The histogram provides the distribution of actual patients. ROC = receiver-operating characteristic.

Discussion

Top Methods Data Collection Results Discussion Author & Article Info References

Two months after hospitalization, one third of the surviving patients interviewed in our study had severe functional limitations. This included one fifth of the patients who entered the study with no limitations in activities of daily living. On the other hand, nearly one third of the patients admitted with four or more dependencies in activities of daily living showed improvement by 2 months. Consistent with findings in previous descriptive studies [28, 35-37], these results indicated that severe functional limitation is a common outcome for seriously ill hospitalized patients and that functional outcomes vary substantially among individual patients. We also found that information obtained from the chart and from a brief interview with seriously ill patients or their families near the time of hospital admission could predict the future functional status of patients. The most important single predictor of serious functional limitations 2 months after hospital admission was functional status 2 weeks before hospitalization (Figure 3). This is not surprising because many of the acute disorders studied in SUPPORT are associated with chronic compromise in functional status and because some limitations are irreversible. The functional status outcome of the patients was also influenced, however, by the nature of the primary disease, by the severity of physiologic derangement, by the patient's quality of life and capacity for physical activity, and by the number of days the patient had been in the hospital before study entry. Age conferred a risk for functional limitations when greater than 60 years.

View larger version (14K): [in this window] [in a new window]   Figure 3. Distribution of limitations in activities of daily living at 2 months with baseline limitations. Numbers above the bars indicate the number of limitations at 2 months. Increases in the proportion of patients with four or more limitations at 2 months are associated with more baseline limitations. ADL = activities of daily living.

Although this is the largest study to attempt to predict functional outcome in severely ill hospitalized patients, other models have been described. Daverat and colleagues [10] used logistic regression to predict "survival with good function or only moderate disability" for 166 patients after intracerebral hemorrhage. Forty-one of the 95 survivors had a satisfactory outcome, and younger age was the most important predictor. Rubenstein and colleagues [8] attempted to predict improvement in functional status among patients admitted to a geriatric evaluation unit. They found that younger age and absence of an unstable medical problem were associated with improved functional status. At baseline, both functional and cognitive status were higher in patients who were discharged to their homes than in those discharged to nursing homes. Chelluri and colleagues [12] found that patients aged 65 to 74 years (n = 12) and 75 or more years (n = 12) who survived 1 year after admission to an intensive care unit had similar quality of life.

The surprising finding was the frequency with which patients either improved or deteriorated. Some of this variation may have been due to the nature of the acute disorder and other clinical factors. Patients with different illnesses may follow different trajectories of functioning. Thus, patients with congestive heart failure may have relatively less risk for long-term limitations because their symptoms may be reversible or alleviated by medications. Survivors of coma may be at greater risk for long-term functional limitations because of the irreversible nature of their deficits. Quality of life soon after hospital admission predicted serious functional limitations at 2 months independent of baseline scores for activities of daily living. This effect may stem from aspects of quality of life that are distinct from functional status or from associations between quality of life and mood, progression of disease, expectations, or motivation that may affect functional status after discharge.

Medical practice, especially acute hospital care, concentrates much effort on aggressive treatment of older patients with preexisting functional limitations and chronic illnesses. Treatments should be considered in the context of the outcomes they are likely to create. It is increasingly recognized that at some point, the burden of therapy may be much greater than the probability of benefit. Models to predict future functioning may help decision makers assess this balance of choosing between an aggressive course of care and optimizing comfort. Concerned parties can also make more realistic plans for support services if prognosis for functional status is available.

Because our predictive model was generated using data from patients who survived 2 months, it must be applied with a survival model such as that described by Knaus and coworkers [14], first estimating the probability of surviving and then estimating the probability that the survivor can function independently. An example of how these data might be presented to a physician or patient is shown in Figure 4, which is the format used for providing information to physicians in the intervention phase of SUPPORT. The conjoint probability of death or serious dysfunction is also shown in Figure 1. However, there are advantages to considering death and functional status separately. Patients vary in the relative value they place on death and various functional states [19]. For some situations, this information can remind patients, families, and physicians of the importance of outcomes other than survival. In other situations, patients already weigh prospects for future function in their decisions about treatment and would welcome the ability to consider functional outcomes more explicitly in their decision making.

View larger version (16K): [in this window] [in a new window]   Figure 4. Phase II feedback form. Sample format presenting prognostic information about survival and functional status to physicians in the intervention phase of SUPPORT. The patient is a 49-year-old man with liver cirrhosis (disease group), and the prognostic estimates are for day 7 of the study. The solid line indicates the predicted mortality rate with 95% CIs. The + sign is the probability of surviving with severe functional limitations 2 months after hospitalization. The probability of not being severely dysfunctional at 2 months if the patient survives is 0.12 (95% CI, 0.06 to 0.25).

Limitations of the Model

Functional outcomes could be examined using operational definitions different from those applied in our study. Although functional status is a continuum, in our initial effort to predict future functioning, we chose to define functional status as a dichotomous outcome. Our simplified presentation (that is, after showing the probability of surviving) shows the probability of having very poor functioning. Dichotomizing scores also facilitated combining scores from the Sickness Impact Profile and activities of daily living scale. However, future studies should examine the usefulness of prognostic models for continuous outcomes that, although more difficult to explain, provide more information.

Finally, our definition of outcome—severe functional limitations—does not capture all of the elements related to evaluations from patients about their continued survival. More than one third of patients who met our definition of severe limitations at month 2 rated their quality of life at that time as "good" or better. This suggests that some patients are satisfied to be alive even though they are disabled [20, 25]. However, others might find lesser degrees of disability to be unacceptable. It may be possible to predict prognosis for quality of life. However, because quality of life is more subjective than functional status, it is easier to use aggregate data to predict a person's functional status than to predict his or her quality of life. In our study, we were limited by the fact that quality of life was measured using a single item, which is inherently less reliable than using a multi-item scale. A prognostic model to predict quality of life should use a multi-item scale to assess quality of life and may need to pay more attention to assessing baseline values or preferences.

The functional outcome of seriously ill hospitalized adults varies substantially. Our model predicted severe functional limitations 2 months after hospitalization on the basis of a small amount of information obtained at the time of hospital admission. Patient reports of previous functioning and their evaluations of quality of life were the most important predictors in the model. Our findings provide a potentially clinically useful technique to estimate both length of survival and likelihood of various functional outcomes. In the future, models such as ours can also compare functional outcomes and quality of care delivered by different hospitals and groups of medical providers.

Presented in part at the 14th Annual Meeting of the Society of General Internal Medicine, Washington, D.C., 29 April to 1 May 1991.