Competing Practice Guidelines: Using Cost-Effectiveness Analysis To Make Optimal Decisions

Methods

In this study, we used optimization to select the best clinical options that, taken together, maximized the number of years of life added to a hypothetical population of 100 000 persons with an age and sex distribution similar to that of the United States in 1991 [19, 20]. For clarity, we considered only a limited number of diseases in the model. (More current clinical guidelines and epidemiologic inputs could easily be used to update our study.)

Selection of Interventions and Clinical Situations

Using MEDLINE to search the clinical literature from 1986 to the present, we sought clinical practice guidelines that 1) addressed clinical situations in which guidelines have actually been used, 2) evaluated differences in outcomes and direct medical costs between or among two or more ways of providing care, 3) used added years of life per patient (unadjusted for quality of life) to measure outcomes of care [21, 22], 4) made recommendations on the basis of cost per unit of outcome [for example, per added year of life], and 5) discounted both costs and outcomes at 5% per year (the “standard” approach to discounting) [23, 24].

Table 1 lists the six interventions that we selected for the model [25-30]. We included examples of major categories of health activities: prevention (prevention of hepatitis B), screening (screening for colorectal cancer), diagnosis (diagnosis of stable angina), risk factor reduction (risk factor reduction for hypercholesterolemia and smoking), and treatment (treatment of recurrent ventricular arrhythmia).

For each clinical intervention, we summarized the key, mutually exclusive directions that could be followed, listing them as clinical options (Table 1, column 2). For the sake of clarity, Table 1 shows only the relevant characteristics of each option; the original [25-30] may be consulted for specifics on such variables as age ranges and doses.

Each clinical option is considered to be of some benefit and is part of the standard repertoire of options that competent physicians might have offered their patients in 1991. Our task was to choose, for a population of patients, a single “best” option for each clinical intervention. We based our selection on total population benefit rather than on benefit for the individual groups of patients for whom guidelines were developed.

Finally, in addition to assuming the existence of a “standard” U.S. population in terms of age and sex, we accounted for the fact that several of the cost-effectiveness studies reported results for multiple, more specific types of patients. Columns three and four in Table 1 highlight the instances in which more than one selection per clinical intervention was needed. For example, where guideline data existed for several types of patients, the model was programmed to select the best option for each type of patient. Selecting a clinical option for colorectal cancer screening, for instance, required only one decision (among the five clinical options) because the reference that we used discussed the group of persons at 65 years of age. However, selecting an option for the diagnosis of stable angina required four decisions, one for each of the clinical subgroups, beginning with those 35 to 44 years of age and ending with those 65 to 74 years of age. We sought the choice of one specific clinical option for each of the 22 different clinical subgroups of patients.

Derivation of Population Cost and Effectiveness Data

For each of the 22 clinical subgroups that required a selection decision, we assigned (from a cluster of allowable options) one baseline clinical option to serve primarily as the standard against which the cost and effectiveness of each competing option could be compared (Appendix Figure 1). Each baseline strategy was considered to be the option most widely “currently practiced.” If the optimization program selected the baseline strategy option for the population-wide “solution,” no additional years of life were added to the population at no additional cost because the incremental cost and effectiveness of a clinical option compared with itself are zero.

For each selection decision, we calculated the incremental cost and incremental effectiveness (in years of life) of each of the allowable alternative clinical options. In other words, we obtained cost and effectiveness data directly from each study in the literature but recalibrated the data, when necessary, to express cost and effectiveness in terms of the baseline option. To determine the incremental cost and effectiveness of each alternative option, we adjusted the data, expressed per patient as obtained from the literature, for the approximate “demand” for that option over the next 12 months in our 100 000-member population. For example, a paper might report that a specific alternative clinical option (compared with the baseline strategy) resulted in a gain of 1.5 more years of life per patient at an additional cost of $20 000 per patient. If the estimated incidence of that clinical condition in our population over the next year was 30 patients, the population-wide incremental cost and effectiveness entries would be $600 000 and 45 years of life, respectively. We expressed all costs in 1991 U.S. dollars [31].

Derivation of Individual Cost and Effectiveness Data

We also identified the most cost-effective clinical option for individual patients within each of the 22 clinical subgroups compared with the baseline strategy, independent of any other selection decision. This option delivered the greatest improvement for each clinical subgroup in years of life per unit cost compared with baseline [32]. For several subgroups, the baseline option was the most cost-effective for those individual persons. For example, if no alternative clinical option provided better effectiveness (more years of life) than the baseline strategy, the baseline strategy was retained as being most cost-effective for the individual patient (Appendix).

Running the Population Model

A spreadsheet was used to summarize all key input variables for the model (Appendix Figure 1). We listed the cluster of clinical options available for each of the 22 clinical subgroups, and we listed the contribution that each alternative option would make to total population cost and effectiveness if it was selected (we determined this by multiplying the estimated population demand for each option with the cost or effectiveness data per patient obtained from the literature).

With these data, we ran the optimization model using “Solver” in Microsoft Excel version 4.0 (Redmond, Washington). We programmed each trial of the model to search for the combination of 22 clinical options (1 for each of 22 clinical subgroups) that maximized population effectiveness (the sum of the effectiveness values of each selected option) while keeping total cost (the sum of the costs of each selected option) at or below a specified constraint that was unique for each trial.

To determine the spacing and extremes of cost constraints, we ran four unconstrained trials in which only total population cost or effectiveness was maximized or minimized. The remaining trials varied by the total cost allowed but sought to maximize effectiveness. The unconstrained trials told us the maximum and minimum possible amounts that could be spent on this constellation of choices. The cost constraints spanned a range of approximately $80 million, from a savings of $45 million to an expenditure of $35 million in our hypothetical cohort.

Results

Table 2 shows the output from one of the 22 cost-constrained trials. (The number of trials is coincidental and unrelated to the 22 different clinical subgroups; we could have produced fewer or more trials by varying the cost constraint). In this example, the cost constraint is $30 million, meaning that the sum of incremental costs could not exceed this amount. The program selected one clinical option for each of the 22 clinical subgroups so that the sum of incremental effectiveness (additional years of life) for the entire population was maximized. Table 2 also shows the contribution of each selected option to population-wide cost and effectiveness.

The model looks separately at the cost and effectiveness of each individual clinical option and makes choices on the basis of total, population-wide cost-effectiveness. In Table 2, for example, 5 of the 22 selected clinical options carried less population-wide benefit compared with the individual baselines. In these instances, the dollars saved probably freed resources to pay for more widespread use of nicotine chewing gum, leading to greater cessation of smoking (as reported in the literature) and therefore longer survival within the entire population. The same can be said about the selection of “no medical treatment” for the hypercholesterolemia intervention. Whenever the most cost-effective option for an individual patient was not selected for the most cost-effective set of interventions for the overall population, it was because not selecting it saved resources, permitting the “purchase” of even more years of life. As expected, the model never selected options that cost more but were less effective than the baseline option; this is consistent with a major tenet of cost-effectiveness analysis [33].

Table 3 shows the results of all cost-constrained trials of the optimization model. The results of these trials are listed in order of increasingly lenient cost constraints. The Appendix Figure 1 lists each option chosen.

Table 3 summarizes the total incremental cost and effectiveness of the set of clinical options selected for each trial of the model. Incremental costs resulting from applying the model ranged from a savings of $45 million to an additional expenditure of $34.2 million; incremental effectiveness ranged from a population-wide reduction of 5077 years of life to a gain of 3238 years of life, depending on the level of cost constraint.

The last column of Table 3 shows, for each trial, the number of instances in which the optimal population-wide solution was also the most cost-effective clinical option for an individual clinical subgroup (that is, for an individual patient). Among all opportunities to select clinical options under different cost constraints, the most cost-effective option for the population was the same as the most cost-effective option for the individual patient only 43% of the time. Of note, this ratio decreased as cost constraints increased.

Discussion

Medical specialty societies, physician groups, pharmaceutical companies, and others have produced more than 1500 sets of clinical guidelines [34, 35]. Furthermore, different organizations may produce different guidelines for the same intervention [36-39]. Lack of uniformity in the design, structure, and presentation of these guidelines has led to the development of rigorous criteria to govern future efforts [40, 41].

Our model evaluates the collective benefit of multiple, competing clinical decision rules, each of which is written for only one type of patient. The model depends on only a few key variables, including estimated population demand for each intervention, the cost per patient of providing the intervention, and the established effectiveness in terms of added survival (years of life). Information on the first two variables can be found in the literature, and data on the third are becoming more readily available. The results of more recent clinical advances and guidelines could be used to update the input choices and values that we used. Moreover, different outcomes, such as assessments of quality of life in affected patients, can be used [21, 22].

Some choices (clinical options) were more robust than others over the model's range of cost constraints. In other words, when multiple trials of a model were run by using a variety of cost constraints, some choices were picked more consistently than others.

For example, for the hepatitis prevention intervention, the choice of screening and vaccinating high-risk newborns and the choice of vaccinating all adolescents was always selected; this is consistent with the conclusion of the original article [25]. For other interventions and clinical subgroups, selected clinical options varied more (Appendix Figure 1).

This observation is important. Some guidelines for individual patients reflect population cost-effectiveness much more than others do. Clearly, a clinical and societal goal should be to use, as often as possible, guidelines that are robust across multiple interventions. Health policy planners should pay attention to such “stability” of individual clinical guidelines if our goal is to use guidelines that are equally beneficial for individual patients and populations wherever possible.

As financial resources become scarcer, the need to understand the balance of the most cost-effective options for individual patients and the entire population increases. In our model, it is notable that as cost constraints gradually increase, the frequency with which the best option for the population matches the best option for individual patients is reduced. This bodes ill for resolution of the tension between the almost infinite demand for health care and a setting of limited resources [7].

We do not suggest that most health care choices can or should be reduced simply to dollars and years of life. The widespread use of optimization technology in industry [14] is due to the availability of good outcomes data (such as information on revenue and expenses) as well as input data. Outcome information on clinical practices is far less available and credible, so it is wise for physician groups, managed care companies, and other decision makers to use optimization and cost-effectiveness analysis only as components of good decisions.

The newly developing health care “industry” must ensure that a proper dialogue between patient and physician remains at the center of each health care encounter. On the other hand, decision making at the population level must also be considered. Skepticism among physicians about cost-effectiveness and decision analysis should be overcome so that both goals-what is best for the individual patient and what is best for the overall population-may somehow be reconciled.

The usefulness of an optimization model and other forms of decision analysis depends on accurate, stable, and credible inputs from guidelines. Interventions done to maximize population benefit are sometimes different from those done to maximize benefit for individual patients. Although there may be significant overlap, the cost of health care interventions reflects more than simply a dollar amount spent; it also reflects what could have been bought but was not [42]. For a society that is rapidly agreeing that it is necessary to simultaneously optimize the costs of health care, the quality of outcomes, and the availability of comprehensive services, we provide a useful framework for better interpreting and selecting from among the recommendations of the numerous clinical practice guidelines.

Populations that enjoy the greatest long-term collective benefit do so because each individual person in those populations surrenders (voluntarily or involuntarily) some autonomy for the greater good. In stable societies, citizens agree to give up some personal freedoms (for example, they observe laws). In economic cartels, member nations agree to limit production to ensure the greatest possible collective profit. In biology, a symbiotic organism sacrifices a totally independent existence for mutual benefits not otherwise easily obtainable. During large-scale casualty situations, the triage process requires forgoing noncritical care to save more lives within the most seriously injured group of patients. The greatest benefit to a community does not necessarily accrue simply by giving each individual member all that he or she needs without regard to the rest of the group [43].

As multiple individual health care organizations compete to achieve market share, tactics may emerge that are more self-seeking than cooperative in terms of optimal population health care, including access, cost, and quality. The guidelines and outcomes data of individual organizations are kept secret and proprietary; clinical coverage policy is often arbitrary and may only loosely reflect scientific knowledge; medical coverage depends on employment rather than being universal. The nature of competition and cooperation in health care suggests that there needs to be more of the latter, less of the former, and greater attention to the common good.

Hardin [44] has described a scenario, the “tragedy of the commons,” derived from the unmanaged use of communal grazing space by early settlers. At some point, the common land runs out of adequate grazing room for all of the animals it supports-individual persons may indeed pursue their own best interests without regard for the welfare of the entire population until resources become limited. The author cautions against “each pursuing his own best interest in a society that believes in the freedom of the commons …” as well as “… the tendency to assume that decisions reached individually will, in fact, be the best decisions for an entire society … .”

Traditional cost-effectiveness research selects guidelines to maximize efficiency within individual clinical subgroups. However, it is essential to consider what is best for society as a whole. To allocate resources as efficiently as possible, decision makers should consider several factors, including the clinical needs of their constituencies, their financial constraints, traditional cost-effectiveness rankings (which evaluate the efficiency of clinical alternatives for individual patients), and cost-effectiveness rankings based on optimal values for an entire population. Finally, the production of “robust” guidelines that are optimal for both individual persons and society can help reduce the need for individual persons to sacrifice clinically optimal strategies for the greater good. However, this is not likely to be easy. As society's resources become scarcer, clinical guidelines selected to optimize population effectiveness reflect individual needs less, making it more difficult for managed care organizations and other resource-allocating decision makers to balance individual and societal priorities.

Appendix

Structure of the Model

The Appendix Figure 1 details the spreadsheet-based structure of the model. The first column lists all six clinical interventions; the second lists the 22 clinical subgroups within these interventions. Each subgroup has a baseline strategy option (given in the third column) and at least one alternative option (given in the fourth column).

Columns five, six, and seven derive incremental cost and effectiveness values. Column five lists each clinical subgroup's estimated demand for each alternative option; this information is derived from demographic and clinical tables on prevalence and incidence rates for specific conditions. For each option, we incorporated this value into the reference-based per-patient cost and effectiveness data to derive population-based incremental cost and effectiveness for each alternative option compared with baseline. (The incremental cost and effectiveness values resulting from choosing a baseline option are zero and are not listed in the Appendix Figure 1)

To program the optimization model, we began by asking the program to assign the integer “0” or “1” to each baseline and alternative option for each clinical subgroup. These values were the so-called decision variables, which were continuously manipulated as the computer sought the best combinations of options for a solution. If any option was selected (by an assignment of “1” rather than “0”), the program multiplied the incremental cost and effectiveness values of the selected option by its decision variable (that is, the value “1”) and placed it in the appropriate column of the solution grid (Table 2).

To complete the programming, we specified the two remaining components of any optimization problem: the objective function and constraints. The objective function is the value to be optimized; in this case, the program was asked to maximize the value of the sum of incremental effectiveness (Table 2). The constraints were specified to enable one and only one option to be selected per clinical subgroup and to ensure that the sum of incremental costs was less than or equal to a constraint specific for each trial run (Table 3). For example, one constraint was written for each cell next to any baseline or alternative option, stating that it could only take on the value of 0 or 1; another set of constraints required that the sums of decision variables for each of the 22 clinical subgroups equal 1, thus ensuring that one and only one choice would be made for each subgroup.

As options were selected, the program continuously tracked the sum of population-wide incremental cost and effectiveness in the solution grid (Table 2). Because selecting any option automatically entered the cost and effectiveness data into the solution grid (through a combination of lookup tables for the verbal description and sum-product formulas for all effectiveness and cost values), there was a direct algebraic tie between the decision variables (0s and 1s); the objective function; and the constraints, including the cost constraint.

Results of Running the Trials

The Appendix Figure 1 (last column) summarizes the options chosen for each of the 22 trials described in Table 3. A shaded box indicates that an alternative option has been selected for a given clinical subgroup during the trial, whereas an absence of shading (white box) for all alternative options in a subgroup of a particular trial indicates the selection of the baseline option. The stability of a specific choice, therefore, can be followed across the range of cost-constraint trials.

To evaluate population-wide solutions, we determined the options that provided the most cost-effective care for each specific clinical subgroup. These are indicated by boldface entries in the Appendix Figure 1 and were derived using the following rules, aided by the “C/E” (incremental cost per year of life) column, which shows the incremental cost-effectiveness of each alternative option compared with baseline.

1. Compare the baseline option with the alternative option or options. We chose the most cost-effective individual option only from among those options that were more effective than the baseline option. If no options had better (that is, positive) incremental effectiveness, the baseline strategy was retained as the most optimal. We did not consider any alternative option with less effectiveness, whatever the cost, as the most cost-effective strategy for individual patients.

2. If any alternative options had better incremental effectiveness, the most cost-effective option for the individual clinical subgroup was that with the lowest incremental cost-effectiveness, evaluated on a continuous scale. Given a C/E of, say, +10 and −15,the latter value would be chosen because this implies higher incremental effectiveness with lower incremental cost (that is, a dominant strategy). If the incremental cost-effectiveness values were all positive (that is, if there was no dominant strategy), the choice was among those options with positive incremental effectiveness and cost, and the lowest of these was chosen. The results show, as expected, that the model never chose an option with positive cost but negative effectiveness.

Dr. Hillman: Leonard Davis Institute of Health Economics, University of Pennsylvania, 3641 Locust Walk, Philadelphia, PA 19104-6218.