The Accuracy of Substituted Judgments in Patients with Terminal Diagnoses

Methods

Patients

Between November 1993 and February 1996, patients were recruited from the outpatient practices of Johns Hopkins Hospital and the Hopkins Bayview Medical Center, Baltimore, Maryland, and Georgetown University Medical Center, Washington, D.C. Eligible patients were older than 17 years of age, spoke English, and had one of the following conditions known to be associated with a less than 50% chance of surviving for 2 years: New York Heart Association class III or IV congestive heart failure with no possibility of transplantation [14], advanced HIV infection according to the Centers for Disease Control and Prevention case definition for AIDS (before the use of protease inhibitors) [15], amyotrophic lateral sclerosis with respiratory compromise [16], unresectable non-small-cell lung cancer [17], or oxygen-dependent chronic obstructive pulmonary disease with dyspnea at rest [18, 19]. Treating physicians were asked to exclude patients whom they thought they were likely to live longer than 2 years and patients who were cognitively incapable of participating. The comparison group comprised general medical patients who were older than 64 years of age, spoke English, and were expected to live longer than 2 years.

To have a power of 0.80 to detect a 15% difference between patient preferences and surrogate predictions at α = 0.05, we needed 263 patient-surrogate pairs. Rounding up to 300 gave a power of 0.75 to detect a 25% difference in agreement between diagnostic groupings at α = 0.05.

Surrogates were chosen according to Maryland's legal hierarchy of surrogates for incapacitated patients: durable power of attorney, guardian, spouse, adult child, parent, sibling, other relative, or friend [20]. For uniformity, surrogates of patients recruited at Georgetown University Medical Center were also chosen according to the Maryland hierarchy.

Attending physicians were contacted 2 weeks before scheduled patient visits. Patients were initially informed of the general nature of the study and asked to supply the phone number of the surrogate who was highest in Maryland's legal hierarchy. At the time of the appointment, details of the study were described and informed consent was obtained. Surrogates were interviewed within 48 hours before or after the patient interview. Patients and surrogates were asked not to speak with each other about the interviews until both interviews were completed.

Informed consent was obtained from all participants. The study was approved by the institutional review boards of the Johns Hopkins Medical Institutions and Georgetown University Medical Center.

Interview Schedule

Structured interviews consisted of closed-ended questions about sociodemographic factors, previous discussions about end-of-life decisions, advance directives, and past experiences. Patients were asked about their treatment preferences, and surrogates were asked to predict patient preferences. Sociodemographic information included age, sex, ethnic group, exposure category for HIV infection, level of education, socioeconomic factors, insurance status, and religious beliefs and practices.

We elicited preferences for various medical interventions in the setting of three hypothetical clinical scenarios that would render patients unable to make decisions for themselves: permanent coma, coma with a small chance of recovery (the “small chance” scenario), and severe dementia (Table 1). For each scenario, the patient was asked to report his or her preferences for intensive care, intubation, cardiopulmonary resuscitation, feeding tube placement, nasotracheal suction, surgery, hemodialysis, esophagogastroduodenoscopy, phlebotomy, and chest roentgenography. All scenarios and interventions were described briefly and simply to ensure understanding. For example, the feeding tube question was worded, “Would you want to be fed by a feeding tube (that is, a tube put through your nose or through the skin over your stomach)?”

Requests for a limited trial of an intervention were interpreted as “yes” responses. Patients and surrogates were urged to give their best judgment. “Unsure” responses were recoded as “yes” responses because they would be clinically interpreted that way. The interview schedules had been previously developed and tested for reliability [13] and were based on the Medical Directive of Emanuel and Emanuel [21]. Slight modifications were made for our protocol; the revised instruments showed high internal consistency, with Kuder-Richardson statistics of 0.97 for the permanent coma scenario, 0.94 for the small chance scenario, and 0.93 for the severe dementia scenario. The complete set of instruments is available from the authors on request.

Surrogate Accuracy in Matching Patient Preferences Scale Scores

We constructed a Surrogate Accuracy in Matching Patient Preferences Scale (SAMPPS) for each of the three scenarios (SAMPPS-1 for permanent coma, SAMPPS-2 for small chance, and SAMPPS-3 for severe dementia). This scale assigns a score of 1 when the surrogate's prediction correctly matches the patient's preference for each scenario-treatment pair. Mismatches are assigned a score of 0. The sum of the correct answers constitutes the SAMPPS score. For each of the three scenarios, a perfect score is 10 and a complete mismatch has a score of 0.

Statistical Analysis

Categorical variables were analyzed by using the chi-square test or the Fisher exact test. Two-tailed t-tests were used for paired continuous variables. Analysis of variance was used to compare continuous variables across multiple groups. Differences between surrogate and patient demographic characteristics were examined by using the McNemar test for paired comparisons. The McNemar test was also used to assess any pattern in mismatches between patient and surrogate preferences. Differences in patient preferences for treatment according to modality or scenario were assessed by using the Cochran Q test.

Although κ scores have often been reported in similar studies, we do not report them because of concern about the appropriateness of κ as a test of surrogate accuracy [11, 22], and the “paradox of κ” [23]. We presented elsewhere an alternative to chi for measuring surrogate accuracy [24].

Factors Associated with the Accuracy of Substituted Judgments

A skewed distribution was found for SAMPPS scores. The Friedman two-way analysis of variance was used to compare SAMPPS scores across scenarios, and the Kruskal-Wallis one-way analysis of variance was used to compare SAMPPS-1 scores across diagnoses.

A binary logit model [25] was constructed using the 10 binary items (correct/incorrect) per patient-surrogate pair of the SAMPPS-1 score. Independent variables were screened for association with SAMPPS-1 score by using the Spearman rank correlation and Kruskal-Wallis tests, as appropriate. Variables that did not contribute significantly to the multivariate model were eliminated in a stepwise manner. When diagnostic categories were stratified by age (≥ 65 years or <65 years), the log odds differed significantly by age group for patients with amyotrophic lateral sclerosis, but no such interaction was found between education and insurance. Model diagnostics to identify influential observations were examined [26], revealing 9 outliers that were subsequently removed from the final model. Thus, the final model reports on only 291 patient-surrogate pairs. The model was validated by using the independent variables found to be significant in the SAMPPS-1 model to predict the SAMPPS-2 and SAMPPS-3 scores.

Results

Participants

Interviews of both patient and surrogate were completed for 75% of pairs that met study inclusion criteria. Of interviews that were not completed, 51% were not completed because of patient refusal; 25% were not completed because of patient refusal to allow surrogate contact; 15% were not completed because of surrogate refusal; and 9% were not completed for other reasons, chiefly our inability to arrange for a surrogate interview within 48 hours of the patient interview. Compared with participants, nonparticipants were older (68 and 60 years of age; P < 0.001), more likely to be female (65% and 39%; P < 0.001), more likely to receive Medicare (24% and 10%; P < 0.001), and more likely to be in the chronic obstructive pulmonary disease or general medicine groups (P < 0.001). Participants and nonparticipants did not differ significantly by ethnic group. Patients and surrogates were interviewed on the same day in all but 31 cases.

Surrogates were younger, better educated, more likely to be married, and more likely to be female than patients (Table 2). They were likely to be spouses (57%) or adult children (22%). The surrogate that we interviewed, chosen according to Maryland's legal hierarchy, was the person the patient would have chosen (the patient's preferred surrogate) in 90% of cases. Patients with AIDS were the youngest, the most likely to be of minority ethnicity, the most likely to have not graduated from high school, and the most likely to lack private insurance (Table 3).

Substituted Judgments

Surrogate accuracy did not differ significantly according to whether patients were interviewed on the same day (SAMPPS-1 score, 7.4) or on different days (SAMPPS-1 score, 7.0) (P > 0.2). Overall, surrogates correctly predicted the wishes of patients for particular treatments in specific scenarios in 66% of instances. Surrogates were better at predicting patients' wishes for more invasive procedures. For example, in the permanent coma scenario, surrogates were correct about ventilator care in 84% of instances and about cardiopulmonary resuscitation in 79% of instances compared with 57% of instances for phlebotomy and 62% of instances for chest roentgenography (P < 0.001).

As shown in Figure 1, accuracy scores did not differ significantly according to patient diagnosis, but they were significantly better for the permanent coma scenario (mean SAMPPS-1 score, 7.4) than for the small chance scenario (mean SAMPPS-2 score, 6.2) and the severe dementia scenario (mean SAMPPS-3 score, 6.3) (P < 0.001 by Friedman two-way analysis of variance).

Surrogates generally chose for patients what they would choose for themselves (in 89% of instances for the permanent coma scenario, 65% of instances for the small chance scenario, and 72% of instances for the dementia scenario), and this preference was usually the same as the patient's preference. For example, of the 51 surrogates who would choose ventilator support for themselves in the small chance scenario, 75% thought that the patient would want the treatment, and 61% of these patients did want it. Of the 249 surrogates who would not want ventilator support themselves, only 37% thought that the patient would want the treatment, and only 36% of these patients did want it.

Mistaken Substituted Judgments

For most modes of treatment, when the surrogates were wrong, they were no more likely (by the McNemar test) to state that the patient would want treatment when the patient did not want it than they were to state that the patient would not want treatment when the patient did want it. The only exceptions across all three scenarios were the less invasive procedures of nasotracheal suctioning, phlebotomy, and chest roentgenography. In the permanent coma scenario, for example, the McNemar statistic was significant only for suctioning (P = 0.02), phlebotomy (P < 0.001), and chest roentgenography (P < 0.001). For these treatments, the surrogate's tendency was to state that the patient would want treatment when the patient did not want it.

Factors Associated with Accuracy

To sort out the weight and independence of factors associated with the accuracy of substituted judgments, we developed a binary logit model by using the responses for the permanent coma scenario as the dependent variable (Table 4). The surrogates of 16 patients with amyotrophic lateral sclerosis who were older than 64 years of age did very well, with an average SAMPPS-1 score of 9.5 out of 10 correct, whereas the surrogates of younger patients with amyotrophic lateral sclerosis and those of patients with congestive heart failure were only about half as accurate as the surrogates of patients in the general medicine reference group. Accurate substituted judgments were associated with patients who had private insurance, patients and surrogates who had high school diplomas, patients who had spoken in detail about their wishes to surrogates, patients who expected to die in 10 years or less, surrogates who never attended religious services, and surrogates who had never had personal experience with a ventilator. Factors that were not significant in univariate analysis or that became insignificant when other factors were adjusted for included patient and surrogate sex, ethnicity, and religious denominations; relationship of the surrogate to the patient; whether the patient had an advance directive; and the surrogates' degree of satisfaction with their understanding of the patients' wishes.

To assess the validity of the final model, which was developed using the SAMPPS-1 responses (permanent coma scenario) as the outcome variable, we substituted SAMPPS-2 (small chance) and SAMPPS-3 (severe dementia) as the outcome variables. The global fit of the model was significant for these two scenarios, with chi-square values of 118.4 (P < 0.001) and 101.9 (P < 0.001), respectively, for −2 log likelihood. All independent variables from the permanent coma model except for the surrogate's personal experience with ventilator treatment and type of patient insurance remained significant in at least one of the two other models.

To discern whether surrogates who attended religious services were “imposing their values” on patients, we explored surrogates' preferences for themselves according to their church- or temple-going behavior. Although numbers were small, surrogates who attended religious services were not significantly more likely to want ventilator treatment for themselves than were surrogates who never attended religious services (13% and 6% for the permanent coma scenario; P = 0.18). However, surrogates who attended religious services were significantly more likely than nonattenders to think that the patients would want ventilator treatment in the permanent coma (17% and 2%; P = 0.007), small chance (46% and 30%, P = 0.04), and severe dementia (39% and 21%; P = 0.02) scenarios.

Practices and Beliefs

Seventy-five percent of patients reported that they had thought about advance directives, and 63% reported having spoken to someone about end-of-life issues. Only 21% of patients who had spoken to someone about end-of-life issues reported that they had spoken to a physician or nurse, and only 44% had spoken with their legal surrogate. For patients who had spoken about their wishes, hospitalization was the cue that prompted this conversation only 4% of the time. Events in the lives of family or friends (47%) were more commonly cited as reasons for such conversations than events in the patients' own lives (36%).

Of patients who had not spoken with anyone about end-of-life issues, 65% intended to do so. Major barriers included not wanting to upset loved ones (41%), not believing that they were ill enough (32%), and uncertainty (30%).

Thirty-three percent of patients had living wills, and 31% had durable powers of attorney for health care. Patients in the general internal medicine group were less likely than patients with terminal diagnoses to have living wills (18%; P = 0.02) or durable powers of attorney for health care (16%; P = 0.01). Of patients who had living wills or durable powers of attorney for health care, 13% and 15%, respectively, reported having learned about these documents from a health care professional. These patients also learned about these documents from lawyers (19% and 22%, respectively), family and friends (29% and 19%), the media (31% and 22%), and other sources (28% and 32%).

Overall, 27% of 249 patients with terminal diagnoses thought that they would be cured. This belief was especially common among patients with unresectable non-small-cell lung cancer (54%) and AIDS (32%) compared with patients with chronic obstructive pulmonary disease (12%), amyotrophic lateral sclerosis (16%), and congestive heart failure (22%) (P < 0.001). Only 10% of the patients with terminal diagnoses judged their own life expectancy to be less than 2 years. Twenty-six percent of patients thought that they would live longer than 10 years.

Preferences

Patient preferences varied according to treatment and scenario. As shown in Figure 2, 13% of patients would choose ventilator treatment if they were in a permanent coma. However, three times as many patients (39%) would choose such treatment if they were told that they had even a less than 1% chance of recovery to their present medical state, and 33% would choose ventilator treatment if they were demented rather than comatose.

Across all scenarios, more patients would choose less invasive procedures (such as chest roentgenography and phlebotomy) than more invasive procedures (such as cardiopulmonary resuscitation and ventilator support) (P < 0.001). For example, in the permanent coma scenario, 20% of patients would want cardiopulmonary resuscitation and 13% would want ventilator support, whereas more than 30% would want chest roentgenography and phlebotomy.

Surrogates were also less likely to prefer ventilator support for themselves in the setting of permanent coma (12%) than in the small chance (17%) or the severe dementia (20%) scenarios (P = 0.003). Across all three scenarios, surrogates were less likely to choose ventilator support for themselves if they were white or never attended religious services.

Discussion

Substituted judgment is an imperfect method for assessing the true wishes of patients who lack decision-making capacity [3-11, 13]. One way of grappling with this imperfection would be to ascertain the conditions under which substituted judgments are likely to be inaccurate so that clinicians can learn when to raise their index of suspicion that patients may be at risk for erroneous substituted judgments. In addition, health care professionals might identify potentially remediable factors associated with poor substituted judgments and design targeted interventions to improve the process. Our study takes initial steps toward these goals.

Substituted Judgments

Our results suggest that the substituted judgments made for patients with terminal diagnoses are not better than those made for general medical patients. These findings show the need for more careful interpretation of substituted judgments for terminally ill patients and for improvements in the process.

In addition to our pilot study [13], two other studies have attempted to elicit factors associated with the accuracy of substituted judgments. The study by Suhl and colleagues [8] was limited by small numbers (50 participants) and a lack of multivariate analysis. The study by Layde and coworkers [11] examined preferences only for cardiopulmonary resuscitation, explored a limited number of independent variables, sampled only hospitalized patients, and was unusual in that 74% of patients wanted cardiopulmonary resuscitation.

Our study corroborates the results of previous studies showing that substituted judgments are more accurate in the setting of an explicit discussion between patient and surrogate about patient preferences [8, 11, 13]. In addition, we found associations with many other factors. For example, more accurate substituted judgments are associated with better-educated patients and surrogates.

As we reported elsewhere [13], the accuracy of substituted judgments is inversely related to the surrogate's church or temple attendance, independent of religious denomination. This counterintuitive finding does not seem to reflect an imposition of the surrogates' wishes for themselves on the patients. Surrogates who attend religious services are actually more likely to report that they do not want the treatment for themselves but would choose it for the patient. Surrogates who attend religious services may be more likely to feel a sense of duty to choose treatment for their loved ones even if they would not choose it for themselves; may be motivated by religiously mediated feelings of compassion, altruism, or guilt; or may be unable to “let go.” We did not ask questions that would allow us to explore these hypotheses.

Optimistic patient estimates of their own prognoses for survival were associated with increased preferences for treatments and inaccuracy in substituted judgments. It is understandable that surrogates might err in their estimates of patient wishes if they have different views of the patient's life expectancy.

Surrogates of patients with congestive heart failure who are not candidates for transplantation made more inaccurate substituted judgments compared with the reference group of general medical patients. Predicting mortality is generally difficult [27], but predictions may be even more difficult for congestive heart failure than for other diagnostic categories because death in congestive heart failure is usually sudden [28]. Perhaps this greater prognostic uncertainty makes it more difficult for patients to make choices or for surrogates to predict those choices. Alternatively, cardiologists may be more reluctant to discuss end-of-life issues because of this uncertainty or for other reasons.

The surrogates of older patients with amyotrophic lateral sclerosis were remarkably accurate. Few patients with amyotrophic lateral sclerosis choose ventilator support [29], and older patients with amyotrophic lateral sclerosis report intense psychological distress [30]. Loved ones of older patients with amyotrophic lateral sclerosis may be especially adept at reading these signals.

It is interesting that, when they were wrong, surrogates were no more likely to say that a patient would want treatment when the patient actually did not than they were to say that the patient would not want treatment when the patient actually did. This suggests that surrogate errors are more random than systematic and may be more amenable to educational intervention.

Patient Beliefs, Practices, and Preferences

As other studies have shown [31, 32], although most patients had thought about end-of-life issues, relatively few had actually prepared advance directives or had spoken to their surrogates about their wishes. Health care professionals were infrequently the source of patient knowledge about living wills or durable powers of attorney.

Terminally ill patients, especially those with unresectable non-small-cell lung cancer and AIDS, made estimates of their own survival prognoses that far exceeded those in the published literature. Despite this optimism, most patients (as other researchers have found [30, 31]) preferred to forgo life-sustaining treatments. Our study provides a more nuanced understanding of this phenomenon by showing that patients with terminal diagnoses are much more likely to forgo invasive procedures than to forgo less invasive procedures.

The clinical scenario also influenced preferences. Almost three times as many terminally ill patients would opt to continue ventilator support if they were told that they had a less than 1% chance of recovery than if they were told that they would never recover from a coma. This finding raises questions about the importance attached to suggesting any hope of survival and about how prognostic estimates are framed [33].

Limitations

Our study sampled only five terminal conditions and may not represent all patients with terminal conditions. We cannot be absolutely sure that “contaminating” discussions did not take place between patients and surrogates, but our data show that accuracy did not differ significantly if patients and surrogates were interviewed on different days. Finally, all scenario studies are somewhat artificial, and it is not known how substituted judgments might change in actual clinical settings. However, this matter can be studied only hypothetically because one cannot know the minds of persons who lack decision-making capacity.

Conclusions and Implications

Reliance on surrogate decisions seems inescapable. Living wills are not universally used, and their specific provisions still require interpretation. If clinicians remain committed to the belief that patient autonomy is not completely abrogated when patients lose their decision-making capacity, there remains little choice but to continue to ask those who know the patient best what they think the patient would want. The information gathered in our study suggests several testable strategies for grappling with this imperfect process.

The accuracy of substituted judgments is associated with multiple clinically apparent patient and surrogate factors. By determining these factors, our study helps clinicians know when to raise their index of suspicion that a patient's loved one may not be representing the patient's true preferences. Under these circumstances, the clinician may wish to take additional steps, such as probing more deeply or involving other relatives.

Several testable strategies for intervention may help surrogates make more accurate substituted judgments. For instance, health care professionals might actively urge patients to talk to their loved ones about their wishes for end-of-life care. Substituted judgments may be improved if clinicians have more frank discussions with terminally ill patients about their prognoses. These interventions might take place in physicians' offices, in community settings, or in support groups for patients with terminal diagnoses. Because surrogate attendance at religious services is associated with inaccurate substituted judgments, community education efforts designed to improve the quality of surrogate decision making should perhaps target churches and temples in particular [34]. Further study is needed to ascertain whether any of the aforementioned strategies will be effective in improving substituted judgments.

Dr. Terry: Johns Hopkins University, Division of Pulmonary Care, 5501 Bayview Circle, Asthma and Allergy Center Building, Baltimore, MD 21224.

Dr. Weisman: Department of Health Management and Policy, School of Public Health, University of Michigan, 109 South Observatory, Room M3138, Ann Arbor, MI 48109.

Dr. Miller: Center for Nursing Research, Johns Hopkins School of Nursing, 1830 East Monument Street, Room 233, Baltimore, MD 21205.

Ms. Stallings: Johns Hopkins School of Hygiene and Public Health, International Health/Human Nutrition, 615 North Wolfe Street, Room 2041, Baltimore, MD 21205.

Dr. Vettese: Johns Hopkins School of Nursing, 1830 East Monument Street, Room 417, Baltimore, MD 21205.

Dr. Haller: Department of Medicine, Johns Hopkins University School of Medicine, 1830 East Monument Street, Room 9061, Baltimore, MD 21205.