Economic Analysis of Health Care Technology: A Report on Principles

1 July 1995 | Volume 123 Issue 1 | Pages 61-70

Preamble

Although economic outcomes research is an evolving field in health services research, there are correct and incorrect ways to conduct and report on economic outcomes studies. Research practices that help to minimize real or perceived bias will increase the quality and usefulness of such studies for those who sponsor, publish, and use them. Because of public concerns about the potential for bias in the design, analysis, and reporting of economic analyses of health care technology, we formed a task force to develop principles to enhance the credibility of these studies. The Task Force on Principles for Economic Analysis of Health Care Technology included participants from academia, the pharmaceutical industry, the public sector, and private research organizations.

As health care resources become increasingly constrained, the information used to make resource allocation decisions must be as reliable, valid, and free of bias as possible. "Getting it right" at the level at which economic results are produced will help to protect consumers and will advance the health of the public.

Bias stems from two broad categories: lack of appropriate independence for researchers and lack of consensus about methods. We focused heavily on the first of these categories for two major reasons. First, few have yet considered the unique issues of researcher independence in economic outcomes research [1, 2]. Second, other investigators have begun to consider and define proper methods for economic outcomes work (an area of considerable controversy) [3-5]. We also looked closely at the requirements for the reporting of economic analyses, which are intended to ensure methodologic transparency and accountability.

The Need for Voluntary Guidelines

Widespread use of economic analysis as part of the development of pharmaceutical, biotechnologic, and medical devices is relatively new. To date, many economic analyses of health care have focused on pharmaceutical agents, and many such studies have been funded by pharmaceutical companies. Results of these economic outcomes studies are used by medical technology firms to support the pricing and marketing of new interventions and to influence national health care systems and third-party payers in their development of coverage and payment decisions. Managed care organizations, hospitals, and government-subsidized health care programs rely on economic analysis of medical technology to help make formulary purchasing and utilization decisions. Physicians may use the results of these analyses to help guide treatment and prescription decisions.

Health care economic outcomes projects are sponsored and conducted by pharmaceutical, biotechnologic, and medical device companies; government agencies; nonprofit foundations; academic investigators; and private research and consulting firms. The standards and methods used to evaluate the safety and efficacy of pharmaceutical products in randomized controlled trials have evolved over 50 years of collaboration among researchers in private, public, and academic settings. Compared with those established to ensure the safety and efficacy of clinical trials, the principles and methods for the conduct of economic studies of health care technology are far less developed.

Problems of conduct, reporting, and bias exist in all types of research [6-14]. In response, codes of conduct, such as those developed by the American Federation for Clinical Research and the Institute of Medicine, have been developed for many scientific disciplines [15-24]. These are good models on which to base principles of conduct for economic outcomes analysis. Although many published principles apply to economic studies, others should be modified and new ones should be developed to guide the conduct and reporting of economic outcomes analyses.

Economic outcomes research requires unique guidelines for the following reasons: 1) As a field, it continues to evolve and is often misunderstood by end users; 2) peer review of it requires special expertise that often exceeds the capabilities of reviewers and scientific journals; 3) it often uses secondary data and requires that many assumptions be made [for example, attribution of a dollar cost to a unit of resource use]; 4) it offers unique methodologic choices, such as which types of costs to include (direct, indirect, intangible, induced), which perspective to apply (that of society, payer, provider, patient), which design to adopt (cost-identification, cost–benefit, cost-effectiveness, cost–utility), from where to obtain costs [indemnity database, managed care or capitated database, hospital cost systems, Medicare, Medicaid], and whether to collect resource consumption data prospectively or retrospectively through various modeling techniques; and 5) economic studies play an increasingly important role in health care decision making because of increasing financial constraints throughout the health care industry. The financial and medical implications of decision making done on the basis of these studies, coupled with the lack of widely accepted guidelines about the conduct and reporting of economic analyses, undermine the credibility of this research. A major issue is that the primary source of funding for this research is often the primary financial beneficiary of positive study results. Unfortunately, even valid studies done under the best of circumstances may be suspect [25-29]. This has led at least one major journal to conclude that these analyses should be viewed much like editorials or review articles are viewed in terms of potential for conflicts of interest [30].

We developed the guidelines reported here after extensive consultation with experts from the public, private, and academic sectors. We recommend that researchers and sponsors adhere to these guidelines, and we suggest they state publicly within their manuscripts that they have done so. End users, including journal editors and readers, consumers, and social decision makers, may then feel more secure in accepting the results of the research, while recognizing that intensive critique of the research will always be necessary.

Operations of the Task Force

The Task Force on Principles for Economic Analysis of Health Care Technology was initiated and organized by faculty from the Leonard Davis Institute (LDI) Center for Health Policy of the University of Pennsylvania. However, because not all LDI faculty were involved, this paper does not represent an official LDI position statement. The Task Force was funded by a coalition of pharmaceutical companies (Appendix A). Funding was also requested (but not obtained) from various government and private foundations. All funding was provided in the form of unrestricted research grants or gifts to the University of Pennsylvania.

Guidelines derive their credibility in part from the composition of the panel that creates them and the process by which they are developed. Candidates for participation in the Task Force from the private sector and the academic research community were identified by the frequency with which they were cited in the health economics literature, which was obtained using a MEDLINE search of literature related to economic analyses of medical technology published between 1983 and 1992.

Approximately 15 members attended each meeting. Minutes were distributed after each meeting and approved by all members present. Members from the sponsoring pharmaceutical companies and the academic organizers rotated so that they numbered three and one, respectively, "at the table" for each formal Task Force meeting. Several professional and governmental organizations, including the Institute of Medicine, the Agency for Health Care Policy and Research, the Food and Drug Administration (FDA), the Health Care Financing Administration, the Centers for Disease Control and Prevention, and a managed care organization, were asked to suggest persons who might participate. In addition, other organizations with a stake in economic analyses were each asked to suggest a person who, because of his or her professional background, had extensive knowledge of or experience with economic analyses. An academic pharmacist, an academic researcher, a private researcher, an ethicist and patient advocate, and an attorney specializing in medical ethics rounded out the Task Force (Appendix B). Two members withdrew from participation. A small "audience" consisting of industry sponsors, academic organizers, staff, and a few other interested parties were invited to each meeting and allowed to comment.

Four formal meetings of the Task Force were held in Philadelphia during 1993 and 1994, and the most substantial work was done by various subcommittees between these meetings. The three main subcommittees were titled "Ethical Conduct," "Responsibility and Control" (the findings of these two subcommittees were later merged into one report), and "Reporting Requirements for Economic Evaluations" (Appendix C). The Task Force was assisted by a professional facilitator. Although Task Force members sought consensus wherever possible on the key issues, consensus was not forced and recommendations were issued only when substantial agreement existed among the members.

Task Force members were not asked to formally represent any organization. Formal endorsement of the final document was not sought.

Copies of this report will be distributed to all organizations that were asked to suggest a participant and to other parties who responded to announcements placed in The New England Journal of Medicine, Scrip, and the Pink Sheet. The Task Force findings will also be reported at academic conferences, medical profession meetings, and appropriate trade conventions.

Findings and Recommendations

Valid approaches to economic analyses can be defined; acceptable methods can be differentiated from unacceptable ones. Bias in economic research stems from two major sources: lack of appropriate independence for researchers and lack of consensus about methods. We suggest ways to minimize bias due to lack of researcher independence, focusing especially on the unique factors that differentiate health care economic outcomes research from other forms of biomedical research. We also include a section on guidelines for the reporting of economic outcomes research as a way of summarizing our work and the work of others in this area.

The principles and recommendations discussed here apply both to research done alone and to research done as a collaboration among persons from industry, academia, or independent organizations. We recommend that scientific organizations adopt these guidelines as minimum requirements for the conduct of economic analyses of health care. Furthermore, we recommend that sponsors, researchers, and editors explicitly indicate their adherence to these guidelines in the economic reports and manuscripts they produce.

Guidelines for Researcher Independence

Existing standards for conducting research are insufficient for economic analysis [6-24]. However, economic research does share many requirements with other types of scientific research, including 1) identification and specification of the research questions; 2) development and specification of the research protocol; 3) negotiation of the funding agreement; 4) definition of how the study team will interact with the sponsor; 5) establishment of procedures for the adoption of protocol amendments; 6) statement of acceptable reasons for termination of the study; 7) specification of how proprietary information will be handled; 8) statement of acceptable reasons to delay the release of results; 9) determination of access to and ownership of data and of rights to further analyze data; 10) clarification of authorship; and 11) determination of the ownership of models and other intellectual property developed during the research. In light of the overlap between economic research and other types of research, we recommend that economic researchers adopt the American Federation for Clinical Research guidelines [22].

Economics research is more credible when investigators adhere to conditions of research that meet generally accepted norms of independence for the researcher, whether or not the researcher is employed by the sponsor. Every effort should be made to conduct research under conditions that minimize the potential for bias that may be introduced by financial and other conflicts of interest. Moreover, given the need for assumptions and choices and the lack of external regulation, trust between researchers and sponsors will probably be enhanced if both acknowledge their responsibility for the integrity of their collaborative work and if both are held to the same set of standards. We focus on the special guidelines required to minimize economic bias through effective researcher independence.

Study Design

In economic analyses, the processes of study design sometimes involve substantial interaction between the research team and the sponsor, and for good reasons. For example, economic study end points in a phase III trial must be compatible with the clinical study end points. Similarly, the capacity of a corporate sponsor to use the results of phase IV studies for promotional purposes is subject to regulatory standards (for example, product uses inconsistent with FDA-approved labeling are not permitted in U.S. promotions).

Protocols should be specified in detail and agreed to by both researcher and sponsor before initiation of any study. Subsequent protocol modifications are acceptable if agreed to by all interested parties before the initiation of data analysis and unblinding (if relevant), and they should be fully documented, explained, and justified in the study report and in publications presenting the study findings. Because economic analyses generally require that some assumptions be made, the protocol should err on the side of assumptions that bias the study against findings favored by the sponsor. Economic studies require that many assumptions and choices be made and may consider both marketed and nonmarketed medical interventions. Although a study can range from being independently designed by the researcher to being designed in a collaborative effort between investigator and sponsor, the research protocol may occasionally be developed solely by the sponsor. Regardless of who designs the study, the principal investigator needs to concur that the study is a comprehensive attempt made in good faith to determine valid economic outcomes of the interventions being studied.

Control of and Access to Data

Researchers should have access to all relevant internal and external documents that may bear on the economic performance of the intervention being tested. Because many economic outcomes research studies involve marketed products, the investigators should have access to all relevant data available about the products, including the results of all previous studies. Health economic projects should never be based on a selective subsample of studies. The research team should retain access to these data to clarify concerns about the research (for example, they may need to do a reanalysis to respond to questions raised about the validity or reliability of the original analysis) and to permit publication of the economic findings if sponsors withdraw funding. Data underlying research findings should be available for review and confirmation, subject to appropriate safeguards to protect data that are legitimately proprietary (for example, chemical production processes) and to protect the privacy of research subjects, unless overriding, legitimate concerns about public health or safety exist. Data should be maintained in accordance with prevailing standards.

Restrictions on access to and control over data are important issues that influence, among other things, whether and what types of future analyses may be done and by whom. In resolving these issues, tensions often arise over access to data, researchers' intellectual property, and protection of proprietary concerns. Use of clinical or economic data for purposes unrelated to the original study objectives and hypotheses or by additional clients may be legitimately restricted by funding agreements (for example, such agreements may require special permission from the sponsor or investigator).

Project Control

Although sponsors may also do economic research for internal purposes, we focus on issues of conduct related to economic research for external use. Regardless of the proprietary interests of the sponsor, an important purpose of research for external use is the discovery of knowledge to improve social welfare. The results of external research are intended for public dissemination, often through peer-reviewed publications.

Control of the conduct, analysis, and interpretation of economic studies should pass to the principal investigator or investigators when the economic study is implemented. However, this does not mean that communication between sponsor and researcher should end. For example, because the sponsor may have the most knowledge about the technical performance of the intervention, researchers may continue to communicate and collaborate with sponsors throughout the study, subordinate to the oversight of the principal investigator.

Reasons for Study Termination

Any valid economic information, especially about products already marketed, can potentially benefit society by helping social decision makers to allocate resources wisely. Therefore, sponsors should not withdraw support for an economic research project because the interim results of the study may not be as favorable as desired (and researchers should not agree to produce such "early looks"). The only legitimate conditions for withdrawal of funding, which will differ depending on whether the product has been marketed, must be specified in the grant or contract. For unmarketed products, legitimate reasons for termination include a halt to clinical evaluation of the product—for example, the study may be terminated if the product will not be marketed.

Financial Relationships between Sponsors and Researchers

Arrangements that reward an investigator personally beyond the real costs of the research should be prohibited; real costs include the market value of the investigator's time, institutional overhead, personnel costs, costs of material needs, and so forth. Economic research done for external use is supported by various funding mechanisms, primarily grants, contracts, and gifts to institutions, each of which generally implies a different amount of appropriate sponsor involvement.

Disclosure

To protect the public interest, economic research requires that sponsors and researchers make accessible to the public all research findings and publications. Because assumptions are necessarily an important part of economic research, each assumption and method chosen should be explicitly disclosed in the final report or publication. Similarly, any choice that may favor the sponsor's product should be carefully and specifically disclosed in the methods section of the publication or report. For example, economic analysis is comparative by nature, and reasons for the selection of comparators should be explained in the context of all relevant comparators.

In addition, standard rules of disclosure apply. Researchers should disclose who supported the research and whether the investigator or investigators were directly employed by or had an ongoing financial relationship, such as a consultancy or stock ownership, with the sponsor. Journal editors should require authors to submit complete comprehensive disclosure statements about any such relationships when they submit a manuscript for review. Investigators and sponsors should make available a description of the financial arrangements between sponsor and investigator, and this description should be published as part of standard disclosure.

Conflict of Interest

Because of the potentially large financial rewards of a favorable economic analysis that assists in acquiring managed care formulary or guidelines committee approval for the intervention under study, conflict of interest (and the potential bias it introduces) is an important issue [25-29]. The same standards apply to all research, whether done by intramural personnel or extramural researchers. Ample opportunities for conflict of interest can arise from the desire for financial gain or professional advancement on the part of either the researcher or the sponsor. Because even the perception of conflict of interest can undermine trust in a research endeavor, efforts should be directed toward avoiding perceived and actual conflict of interest.

Investigators bear a specific responsibility to disclose to potential sponsors any arrangements with other entities that might result in conflicts of interest. An investigator being asked to assess the cost-effectiveness of two competing interventions by two different sponsors should make the arrangement clear to all involved.

Pilot and Feasibility Studies

Pilot and feasibility studies (whether models or primary data collection) should not be used as a way to "peek" at early findings and terminate a study prematurely; they should be separate and distinct from the full study. If data from these studies are to be combined with those of the full study, this should be specified a priori, explained, and justified. Preliminary studies are acceptable only to test whether a project is technically feasible or to develop an appropriate research design or instrument. This is especially important when economic projects are designed to evaluate the effect of already marketed products, in which case any new economic knowledge may be useful to consumers.

Data Analysis and Reporting

The principal investigator has primary responsibility for ensuring the quality and integrity of analysis and interpretation of the data and for presenting and disseminating the findings. Economic conversions (for example, how the number of physician visits are translated into dollar amounts) should be clear to readers, whether or not models were used. Selective presentation of data, such as that which supports a new intervention, should be avoided. Authors should insist that editors allow them to include sufficient information (for example, the main points and assumptions) in the presentation of the study methods so that the validity of the results can be objectively assessed. They should also keep on file and make available to the public (through publication of addresses for acquisition) records of the methods and data not contained in the publication.

Publication and Dissemination Issues

Important findings should be reported completely and in a timely manner, regardless of the results. However, the timing of the release of economic studies is important because positive results may have a significant effect on product pricing, formulary approval, and other factors. In some cases, limited delay in the release of findings may be warranted (for example, to allow for sponsor review of the accuracy of clinical data). Any such conditions should be specified and agreed upon in the original funding agreement and fully disclosed at the time of release.

As in clinical research, researchers employed by the sponsor may control or share control of an economic outcomes study. In such cases, it is imperative that the researcher conduct the study in a manner that follows the standards for independent disclosure, reporting, and dissemination described in the document. Sponsors may recommend places where manuscripts and abstracts should be submitted for publication or presentation. The final decision should be made by the principal investigator.

Authorship

Final determination about authorship issues is the responsibility of the principal investigator. However, specific guidelines and criteria for authorship should be addressed in the funding agreement and should conform to currently established institutional and editorial policies for scientific research [7]. Both extramural and intramural investigators who make substantive contributions to the conceptualization, intellectual development, analysis, and interpretation of the research should be included as authors and should accept responsibility for the results and conclusions of the study. This will affect where the research can be submitted for publication (for example, the policy of The New England Journal of Medicine, which prohibits publication of manuscripts whose authors obtained direct financial support from the sponsor of the study) [30].

Guidelines for Reporting Economic Outcomes Research

The following recommendations are intended to make it easier for readers of economic evaluations to understand 1) the sources of information and methods used to measure and estimate effects; 2) whether an analysis is scientifically sound and relevant to the reader's needs; 3) the validity of conclusions; and 4) the applicability, generalizability, and limits of the reported results. Because the methods and data sources for estimates of the pertinent variables used in economic research vary substantially, decisions made by the investigator about methods can greatly affect study results. Mathematically accurate results can be produced regardless of the choices and assumptions designed into the economic analysis [31]. Therefore, it is vital that consumers of economic research be able to understand fully the methods used in these studies and their implications.

Consistent with standard scientific principles, economic outcomes studies should be reported in sufficient detail and with sufficient clarity that readers can both understand and replicate the methods used. Because of the lack of standardization of economic methods and the uncertainty inherent in the estimation of many variables included in economic models, authors of reports on economic studies have an additional, intensified responsibility to provide readers with a rationale for the methodologic approaches and information sources used in the study. Researcher independence must be established and shown. Reduction in actual and perceived bias will help to produce sound studies of high quality. The following topics should be addressed in reports of and journal articles on economic research.

Background, Study Purpose, and Rationale

The purpose, hypothesis, and rationale of the study should be explicitly stated at the beginning of the presentation. Primary and secondary goals should be clearly identified. Pertinent clinical, economic, and methodologic literature should be cited and appraised. In introductory sections, the investigator should provide a description of current clinical practice in management of the condition under study and discussion of the known effects of all relevant therapies. This information gives readers a context and background for understanding the goals of the study, the rationale for the study design, the analytic methods chosen, and the decisions made in doing the analysis. Particular attention should be given to any previous evaluations on the same topic. Strengths and limitations of the existing literature should be highlighted.

Choice of Comparators

Although many clinical trials compare an intervention with a placebo, economic studies of new therapies generally seek to estimate the marginal (incremental) as well as the average costs and outcomes of new therapies relative to those currently used in medical practice. To make a fair and relevant comparison, one must consider the intensity of a treatment and its appropriate clinical alternatives, including, where appropriate, nonpharmacologic treatment and no treatment. In practice, it may not be possible to compare all potentially relevant therapeutic options. Comparators selected might include the least expensive currently available strategy, the most commonly used alternative therapy, and the most effective currently available interventions. Inappropriate choices and misleading justifications of comparators may bias a study. Thus, the report should provide a discussion of current pharmaceutical and nonpharmaceutical alternatives and justification of the comparators selected for the study.

Perspective

The perspective taken in doing the research (for example, the viewpoint of society, patients, third-party payers, or particular providers) should be reported and justified. In general, the societal perspective is the broadest and the most appropriate for general decision making because it leads to optimal decisions. However, other perspectives are also valid. Readers advised of the perspective taken in the analysis can better interpret results and evaluate the appropriateness of other methodologic decisions made by the investigator, such as the inclusion or exclusion of certain costs. Where data limitations or other factors cause the actual methods used to be inconsistent with the stated perspective (for example, if a study is said to be done from a societal perspective, but drug "costs" are measured by wholesale rather than retail prices), these deviations should be highlighted and their implications discussed.

Data Sources

Readers should be able to understand the methods of data collection. The sources for the study data should be documented, particularly for modeling studies in which multiple data sources may be required. Each data source should be explained in sufficient detail so that readers can evaluate the strengths, weaknesses, and possible sources of bias that may be inherent in the data used in the analysis. Selection criteria for studies and databases should be described. Authors should explicitly note the direction and magnitude of potential bias in their data sources.

Study Design

The researcher should describe the methods used and discuss their advantages and limitations. The type of analysis being done (for example, cost-identification or cost-minimization, cost–utility, cost-effectiveness, or cost–benefit) should be explained.

The study design with the highest internal validity for evaluating both clinical and economic effects is the randomized controlled trial. However, when effectiveness rather than efficacy is of interest, when event rates of interest are extremely small, or when the time horizon for health and economic effects is much longer than the time required to achieve clinical effects (see Time Horizons), a randomized controlled trial may not be practical, desirable, or ethical. In such cases, other study designs, such as a pre-post approach, a large-sample trial, modeling, a case–control study, a multivariate analysis, use of comparison group method, a decision analysis, or use of simulation method, can be useful in estimating costs and outcomes. These methods have weaknesses; for example, they may not permit the analyst to discern the effects of preexisting conditions from the technology being studied because of selection bias and other factors. It is important for readers to be able to assess the extent to which study designs support the validity of the data sources that underlie study conclusions. Authors should discuss the strengths, weaknesses, and limitations of data sources regardless of whether the study investigators collect original data, rely on historical data, or review the literature.

When modeling or other procedures are used to estimate health outcomes and resource utilization, the logic underlying the choice of the model and the analytic techniques used should be described. A discussion of the potential limitations and implications of these modeling methods relative to a fully experimental design should also be included. The procedures used to estimate outcomes (for example, the number of cases of stroke averted) and to value outcomes (for example, the cost of treating a stroke) and effects on quality of life should be described in sufficient detail so that readers can replicate the methods used. Assumptions should be described and potential limitations of the estimation procedures should be noted (such as failure to model long-term side effects and their associated costs).

Measurement of Costs

The report should describe all relevant costs and discuss how they were valued and included in the analysis. Types of relevant costs include direct, indirect, and intangible costs.

Direct costs include all substantial costs (and cost offsets) of the intervention, including relevant personnel, supply, and facility costs (or savings) involved in the administration of the intervention. Direct nonmedical costs (for example, expenses for such illness-related items as special food or clothing, transportation, or lodging), which are usually borne by patients, should be included in the study when pertinent to the perspective. Future medical costs likely to be incurred as a result of the intervention and comparator treatment ("induced costs") should be addressed. Resource use omitted because of small magnitude or extreme difficulty in collection should be explicitly noted.

Indirect costs relate to the cost of lost productivity due to a health care intervention. Lost productivity is usually valued as lost wages or an imputed monetary value of time. Indirect cost or productivity losses caused by the intervention should be contrasted with the indirect costs of illness. Reduction in the indirect costs of illness is often estimated as a monetary benefit, especially for cost–benefit studies (see Indirect Costs and Benefits). Some studies also include such factors as increased social security payments as a result of prolonged survival. Although these may be important from some perspectives, they should be described as transfer payments and not as economic costs. These costs may or may not be included in an economic outcomes analysis. Reasons for inclusion or exclusion should be provided.

Intangible costs represent an attempt to assign a dollar value to consequences such as pain, suffering, and grief. These costs are implicitly, and sometimes explicitly, contained within the quality-adjusted life year when it is calculated and within the benefit if the study is a cost–benefit analysis that uses the willingness-to-pay method (see below).

Readers should be able to easily distinguish between cost offsets (reductions in the costs of services such as hospitalization and physician visits that result from the benefits of the intervention) and the direct costs of therapy. They also should be able to understand how individual resources have been combined into categories if categories have been used. They should also be able to distinguish costs by categories because each category has strengths and weaknesses and should be able to facilitate comparisons with other studies (for example, they should be able to separate indirect from direct costs and direct medical from direct nonmedical costs).

Outcome Measures

All important outcomes, including short- and long-term untoward effects of the intervention such as major complications, should be incorporated as fully as possible. However, because of limited data availability and resource and time constraints, some potentially important outcomes cannot be measured in some cases. These omissions should be identified. The reasons for and the implications of these omissions should be stated and explained.

In general, choosing appropriate health outcomes is an important task in economic analysis that depends on the type of study being conducted.

In a cost-effectiveness analysis, outcome measures related to morbidity and mortality are reported in "natural units" such as years of life saved, incident cases, and functional status (such as disability, including bed days, work loss days, and usual activity days) rather than converted into dollars.

In a cost–utility analysis, outcomes are converted to standard metric measurements. Typical outcome measures are time-weighted quality of life and utility measures, such as quality-adjusted life years measured by any of several methods.

In a cost–benefit analysis, all outcomes are converted into monetary units. Outcomes or benefits include reductions in the indirect costs of illness, such as productivity gains resulting from the therapy (see Indirect Costs and Benefits).

Some outcome measurements require special explanation and justification. First, quality of life and utility measures should indicate the basis of the unit of measurement (for example, how utilities were derived). This is particularly important when the primary outcomes that need to be compared differ across interventions or diseases. The measurement basis is also important when monetary and natural units do not adequately capture health status changes (for example, if a simple intervention has many important, clinically relevant outcomes). If such measures cannot be included or estimated, careful explanation of the implications this has for the value of the study is necessary. Second, because their implications are easily understood, natural unit measures (for example, morbidity and mortality) are important, even when monetary or quality-of-life measures are the primary outcomes. Additional information about effects on morbidity and mortality and intermediate measures (surrogate end points) should be available to provide readers with sufficient background and to support the use of these measures and the other outcomes included. Third, intermediate measures (such as changes in blood pressure or smoking cessation) are sometimes used for economic analyses. The evidence of the strength of such links between intermediate outcomes and final health outcomes should be provided and discussed. Although evidence may change over time, decisions must often be made despite uncertainty (see Sensitivity Analysis).

Quality-of-life and functional status measures reflect different effects of illness and its treatment on a patient and are in common use. However, reports should include discussion of the potential limitations and implications of the specific functional status adjustments and measurements used.

Indirect Costs and Benefits

Investigators should carefully describe the reasons for the inclusion and exclusion of indirect costs and benefits and how these are valued, if included. Especially in cost–benefit studies, these choices can cause vast differences in final results. Therefore, if indirect costs are included in an analysis, they should be presented so that the reader can separate these estimates from the direct costs in the final analysis.

Controversies exist about whether and how indirect or intangible costs and benefits should be included in studies. Specific controversies include actual versus potential productivity, market versus nonmarket productivity, and human capital versus willingness to pay.

Actual versus potential productivity: Studies, particularly those taking the societal perspective, may measure indirect costs or benefits as the value of potential productivity lost or gained (for example, the total potential productivity gained if heart disease-related mortality were reduced). It is also argued, however, that if the economy is not at full employment, the true indirect costs or benefits may only be the costs associated with replacing or retraining workers. The choice between these two methods of evaluating indirect benefits would probably lead to different results. Authors should justify their choices, particularly with reference to the study perspective.

Market versus nonmarket productivity: Some studies estimate indirect costs and benefits only for persons who are employed (market-related productivity). It is often argued that this method biases results toward interventions affecting demographic groups with high participation in the labor force. Other studies use various methods to impute a productivity value to activities outside the labor market, such as housekeeping services. Again, this choice should be explained and contrasted with its alternatives.

Human capital versus willingness to pay: The estimation of productivity-related costs and benefits is generally known as the human capital method. It involves the calculation of the net present value of the future stream of a person's lifetime earnings that could have been expected if the person had not died or become disabled prematurely. The human capital method is often contrasted with the willingness-to-pay approach, which is based on the fundamental assumption that an individual person's preferences or priorities should be considered and that these preferences and priorities can be expressed in monetary units. Theoretically, the willingness-to-pay approach is preferred because it is a broader measure of the benefits related to an intervention: It includes productivity, quality of life, and other intangible factors. In general, studies should include all costs and benefits relevant to the perspective taken. In either case, authors should clearly explain the implications for the results of the willingness-to-pay approach compared with the human capital approach.

Valuation of Costs and Outcomes

Estimates of costs and any monetary outcomes should be reported as the quantity of each resource consumed and the unit price of each resource. The report should show the appropriateness of the unit price data to the perspective of the study (for example, accounting costs and out-of-pocket costs are more appropriate for a study that uses the societal perspective, whereas actual payments are more appropriate for a study that uses a payer's perspective). Whether national or local, indemnity or managed care, and cost or charge data are being used should also be reported.

Monetary and nonmonetary costs and outcomes incurred in the future should be discounted (often at 5%). If they are not, the report should explain why. Analyses of the sensitivity of conclusions to the discount rate used should be done and the implications discussed [31].

Time Horizons

For the reader to assess whether all relevant costs and outcomes related to a therapy have been considered, the time horizon for the study should be described and justified. Although many clinical effects may be achieved within a short time, other costs and benefits may occur only over a long period.

The time horizon selected for an analysis should be relevant to the clinical condition and the expected effects of treatment. Sufficient information should be provided so that the reader can evaluate the time frame used for estimating cost and benefits relative to the hypothesized time sequence of costs and outcomes.

Sensitivity Analysis

This involves modifying estimates of parameters over a plausible range of values. Sensitivity analysis allows readers to judge the effect on study results of alternative assumptions for the range of potential values for uncertain parameters. These analyses should include evaluation of the full range of values included in the confidence intervals or other rough estimates for critical variables for which common statistical measures of variability are available. Sensitivity analyses should also be done over the range of potential assumptions for other variables (both epidemiologic and economic) for which limited data are available. The particular variables selected for sensitivity analysis and the form of sensitivity analysis used (for example, investigators may permit one element to vary at a time or may permit several elements to vary simultaneously) should be described and justified so that readers can determine the robustness of the study results.

Generalizability and Limitations

Projecting study conclusions to the universe of patients, settings (which may influence the behavior of those who administer and receive treatment), and times (future effects) involves recognizing and adjusting for potential differences between study circumstances and the way interventions will be used. The data and methods used to do the analysis and the limits on the generalizability of the results to other populations of patients, settings, and times should be fully described. In particular, studies should consider the difference between efficacy as shown in clinical trials and effectiveness, defined as the benefit derived from the therapy when used in usual circumstances. Editors of medical journals should be strongly encouraged to permit full discussion of key assumptions and methods in articles reporting on economic studies.

Peer Review and Responsibilities of Journal Editors

Editors should expect that articles reporting on economic analyses may require more space than has historically been allotted. The articles should present all of the main methods and assumptions as discussed in this report. Appendixes should be resorted to only if necessary, and further detail should be available from the authors. Editors should identify and use reviewers with a good working knowledge of the economic outcomes field to evaluate these manuscripts.

Appendix A

The pharmaceutical companies that provided financial support through academic grants to the University of Pennsylvania included the following: Amgen, Inc.; Astra USA, Inc.; Genentech, Inc.; Lederle Laboratories; Merck & Company, Inc.; Ortho Biotech, Inc.; Rhone-Poulenc Rorer Pharmaceuticals, Inc.; Sandoz Pharmaceutical Corporation; Searle; SmithKline Beecham; Sterling-Winthrop, Inc.; and the Upjohn Company.

Appendix B

The following are organizers of the Task Force on Principles for Economic Analysis of Health Care Technology:

Alan L. Hillman, MD, MBA *, Director, Center for Health Policy, Leonard Davis Institute of Health Economics; Associate Professor of Medicine and Health Care Management, School of Medicine and The Wharton School, University of Pennsylvania.

———————————————

*: Organizers who rotated as members during the four formal meetings (one organizer was present at each meeting)

J. Sanford Schwartz, MD *, Executive Director, Leonard Davis Institute of Health Economics; Robert D. Eilers Professor of Medicine and Health Care Management and Economics, University of Pennsylvania.

———————————————

*: Organizers who rotated as members during the four formal meetings (one organizer was present at each meeting)

Mark V. Pauly, PhD *, Director of Research, Leonard Davis Institute of Health Economics; Bendheim Professor, Professor of Health Care Systems, Public Policy and Management, Insurance and Risk Management, and Economics, University of Pennsylvania.

———————————————

*: Organizers who rotated as members during the four formal meetings (one organizer was present at each meeting)

Bernard S. Bloom, PhD, Research Professor of Dental Care Systems, Psychiatry and Health Care Systems, University of Pennsylvania.

John M. Eisenberg, MD, Chairman and Physician-in-Chief, Department of Medicine, Georgetown University Medical Center.

Mary Kaye Willian, DrPH, Project Director for the Task Force, University of Pennsylvania; Senior Fellow, Leonard Davis Institute of Health Economics; Senior Research Investigator, School of Medicine.

The following are nonindustry members of the Task Force on Principles for Economic Analysis. Organizations are listed for purposes of identification only. These members were present "at the table" for all or most of the four formal Task Force meetings.

Molla Donaldson, Senior Staff Officer, Division of Health Care Services, Committee on Potential Conflicts of Interest, Institute of Medicine, 2101 Constitution Avenue, Washington, DC 20418.

Allan Lazar, Deputy Director, Center for Research Dissemination and Liaison, Agency for Health Care Policy and Research, 2101 East Jefferson Street, Suite 501, Rockville, MD 20852.

Sheila Leatherman, MA, President, Center for Health Care Policy and Evaluation, United HealthCare Corporation, P.O. Box 1459, Minneapolis, MN 55440-1459.

Bryan R. Luce, PhD, MBA, Director, Battelle-Centers for Public Health, Research and Evaluation, 2101 Wilson Street—Suite 800, Arlington, VA 22201.

Barbara Mishkin, Esquire, Hogan and Hartson, Columbia Square, 555 Thirteenth Street, NW, Washington, DC 20004-1109.

Louis A. Morris, PhD, Chief, Marketing Practices and Communications Branch, Division of Drug Marketing, Advertising and Communication, Food and Drug Administration, 5600 Fishers Lane HFD-246, Rockville, MD 20857.

Gail Povar, MD, MPH, Department of Health Care Sciences, George Washington University, 2150 Pennsylvania Avenue, NW, Washington, DC 20037.

Stephen W. Schondelmeyer, PharmD, PhD, Professor and Director, PRIME Institute, University of Minnesota College of Pharmacy, 7-159 Health Sciences Unit F, 308 Harvard Street, SE, Minneapolis, MN 55455.

John Schrogie Jr., MD, Assistant Director, Health Policy and Clinical Outcomes, Thomas Jefferson University Hospital, 1015 Walnut Street, Philadelphia, PA 19107.

Steven Sheingold, PhD, Director, Technology and Special Analysis Staff, Health Care Financing Administration, Room 100, East High Rise, 6325 Security Boulevard, Baltimore, MD 21207.

Earl Steinberg, MD, MPP: Director, Program for Medical Technology and Practice Assessment, Johns Hopkins University, 1830 East Monument Street, Room 8068, Baltimore, MD 21205.

Steven M. Teutsch, MD, MPH, Chief, Prevention Effectiveness Activity, Epidemiology Program Office Centers for Disease Control and Prevention, MS-F53, Atlanta, GA 30333.

Feedback from journal editors (who were not formal members of the Task Force) was obtained throughout the deliberations.

The following are industry-affiliated sponsors who rotated attendance so that three were present "at the table" at each of the four formal Task Force meetings: Others were also given access to these meetings as observers.

Marc L. Berger, MD, Director, Outcomes Research and Management, Merck & Company, Inc., P.O. Box 4, WP39-105, West Point, PA 19486.

Joseph Crawley, MS, Manager, Product Surveillance and Epidemiology, Lederle Laboratories, Building 190, Room 306B, 401 North Middletown Road, Pearl River, NY 10965.

Jacob Drapkin, Director, Reimbursement and Health Economic Systems, Ortho Biotech, Inc., 700 Route 202, South Raritan, NJ 08869.

Haim Erder, PhD, Manager Health Economics, Amgen, Inc., Amgen Center, 17-2-C-391, 1840 DeHavilland Drive, Thousand Oaks, CA 91320.

Robert A. Freeman, PhD, Director, Pharmacoeconomics, Sterling-Winthrop, Inc., 90 Park Avenue, New York, NY 10016.

Lloyd P. Haskell, MD, Senior Director, Clinical Research and Outcomes Management, Astra USA, Inc., 50 Otis Street, Westborough, MA 01581.

Hind T. Hatoum, PhD, Director, Pharmacoeconomics, Corporate Product Planning, Searle, 5200 Old Orchard Road, Skokie, IL 60077.

Maarja Hildebrand, Manager, Cost Benefits Studies, National Accounts & Managed Care, SmithKline Beecham, FP 1205, One Franklin Plaza, P.O. Box 7929, Philadelphia, PA 19101.

Todd Rich, MD, MBA, Senior Manager, Health Economics, Genentech, Inc., 460 Point San Bueno Boulevard, Mail Stop 58, South San Francisco, CA 94080.

Theodore E. Spiro, MD, Director, Clinical Research, Cardiovascular Diseases, Rhone-Poulenc Rorer Pharmaceuticals, Inc., Central Research, 500 Arcola Road, Collegeville, PA 19426.

Richard J. Willke, PhD, Health Economics Scientist, Epidemiology and Pharmacoeconomics, The Upjohn Company, 7000 Portage Road, Kalamazoo, MI 49001.

Appendix C

The following are members of the Writing Committees of the Task Force on Principles for Economic Analyses:

Responsibility and Control: Molla Donaldson, Bryan R. Luce, (Writing Committee Chair), J. Sanford Schwartz, Theodore E. Spiro, and Richard J. Willke.

Reporting Requirements for Economic Evaluations: Marc L. Berger, Haim Erder, Louis A. Morris, Mark V. Pauly, Steven Sheingold (Writing Committee Chair), Earl Steinberg, and Steven M. Teutsch.

Ethical Conduct: Robert Freeman, Maarja Hildebrand, Sheila Leatherman, and Gail Povar (Writing Committee Chair).

The following persons integrated the final report on behalf of the Task Force: Marc L. Berger, Alan L. Hillman, Bryan R. Luce, Gail Povar, Steven Sheingold, J. Sanford Schwartz, and Mary Kaye Willian.

Glossary

Top Glossary Author & Article Info References

Bias: A range of factors that systematically influence the measures undertaken independent of the studied intervention; a tendency, intentional or unintentional, to inappropriately or unfairly favor one or more of the interventions being evaluated. It can affect the formulation of study questions, study design, data collection, data analysis, or presentation of results.

Clinical trial phases of drug development: The 1962 amendments to the Food, Drug and Cosmetics Act require a sponsor to apply to the FDA for permission to initiate human testing with an Investigational New Drug.

Phase I studies are the first to involve humans and are intended to provide information on the dose of an experimental drug that might be used, how often it should be given, and its potential side effects.

Phase II studies emphasize the efficacy of a compound in treating a given clinical condition. Most are double-blinded, randomized, controlled clinical trials that require the choice of clinical end points.

Phase III studies may involve several thousand patients, are usually multicenter trials, and are often multinational in scope. On average, they last between 1 and 4 years. They are designed to clarify a compound's therapeutic effects and provide information on side effects and possible toxicity; they are the third phase of the evaluation of a compound before application for approval by the FDA.

Phase IV studies are done after the FDA approves marketing a compound; they are sometimes called postmarketing studies. Information about rare or delayed side effects and long-term effectiveness is collected in phase IV studies. They may be used to develop data for a new approval and changes in drug labeling.

Conflict of interest: A situation in which a person subjects herself or himself to competing or potentially contradictory duties to more than one party such that meeting obligations owed to one party violates or risks violating obligations owed to another.

Contract: An award of funds (for example, to support research). It is generally used to describe an agreement in which a product is procured for the use of the sponsor. Contracts usually provide less flexibility and are more directive and restrictive than grants.

End points: The health and economic outcome variables measured. Different clinical end points might be measured during different phases of technology development and before and after approval. Primary and secondary end points of a trial should be specified in the protocol before initiation of analysis and unblinding (if relevant). The overall success of the trial is usually gauged by one or two primary end points and by more secondary end points that are associated indicators of the safety, efficacy, or effectiveness of the treatment.

Grant: An award of funds (for example, to support research) in which a study or area of investigation is indicated but in which an exact product to be delivered is not specified. Research grants are more open-ended and flexible and less restrictive and directive than contracts. The cooperative agreement is another form of assistance in which sponsor and research are expected to have considerably more interaction than they would in a grant relationship.

Nonapproved indications: Those uses of an approved compound that have not been approved by the FDA for labeling.

Outcomes research: Encompasses the entire field of research involving study end points that are meaningful to patients, including quality of life, death, or dollars expended for treatment. Economic analysis is a subset of outcomes research with study end points focusing on monetary outcome measures, such as costs of treatment. Pharmacoeconomics research is a subset of economic analysis that focuses on the evaluation of pharmaceuticals.

Researcher: Any scientist or investigator, whether employed by a sponsor or not, who directs or directly contributes to the conceptualization, design, and implementation of studies. A researcher may be employed by an academic health center, a research organization, a government agency, a medical products company, or an insurer or may be a community-based practitioner working with one of these entities.

Sponsor: One who funds research. Sponsors may include medical products companies, health care organizations, the government, or philanthropic foundations.

Author and Article Information

Top Glossary Author & Article Info References

Requests for Reprints: Alan L. Hillman, MD, MBA, Leonard Davis Institute of Health Economics, Center for Health Policy, University of Pennsylvania, 3641 Locust Walk, Philadelphia, PA 19104-6218.
Disclaimer: Some of the views expressed in this report do not necessarily reflect those of all individual members of the Task Force or their affiliated organizations. Individual members did not officially represent the affiliated organizations.
Disclosure: The public announcement (at the time of a presentation or publication) of the evaluation and interpretation of the results of a study. Disclosure includes discussion of sources of possible bias, potential conflict of interest, relevant constraints imposed on investigators in their conduct or reporting of the study, the nature and amount of support received by the investigators, and other pertinent information necessary for full evaluation of the report.

References

Top Glossary Author & Article Info References

1. Henry D. Economic analysis as an aid to subsidization decisions: the development of Australian guidelines for pharmaceuticals. PharmacoEconomics. 1992; 1:54-67.

2. Detsky A. Guidelines for economic analysis of pharmaceutical products: a draft document for Ontario and Canada. PharmacoEconomics. 1993; 3:354-61.

3. Udvarhelyi IS, Colditz GA, Rai A, Epstein AM. Cost-effectiveness and cost–benefit analyses in the medical literature. Are the methods being used correctly? Ann Intern Med. 1992; 116:238-44.

4. Luce BR, Simpson K. Methods of cost effectiveness analysis: areas of consensus and debate. Clin Ther. 1995; 17:263-80.

5. Drummond M, Brandt A, Luce B, Rovira J. Standardizing methodologies for economic evaluation in health care. Practice, problems, and potential. Int J Technol Assess Health Care. 1993; 9:26-36.

6. Freedman B. Equipoise and the ethics of clinical research. N Engl J Med. 1987; 317:141-5.

7. Uniform requirements for manuscripts submitted to biomedical journals. International Committee of Medical Journal Editors. JAMA. 1993; 269:2282-6.

8. Angell M, Relman AS. Ethical imperialism? Fraud in biomedical research: a time for congressional restraint (Editorial). N Engl J Med. 1988; 318:1462-3.

9. Engler RL, Covell JW, Friedman PJ, Kitcher PS, Peters RM. Misrepresentation and responsibility in medical research. N Engl J Med. 1987; 317:1383-9.

10. Chalmers TC, Frank CS, Reitman D. Minimizing the three stages of publication bias. JAMA. 1990; 263:1392-5.

11. Sharp DW. What can and should be done to reduce publication bias? The perspective of an editor. JAMA. 1990; 263:1390-1.

12. Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA. 1990; 263:1385-9.

13. Chalmers I. Underreporting research is scientific misconduct. JAMA. 1990; 263:1405-8.

14. Schafer A. The ethics of the randomized clinical trial. N Engl J Med. 1982; 307:719-24.

15. U.S. Public Health Service Panel on Cost-Effectiveness in Health and Medicine, Washington, D.C.: U.S. Public Health Service 1995.

16. American Federation for Clinical Research National Council. American Federation for Clinical Research guidelines for avoiding conflict of interest. Clin Res. 1990; 38:239-40.

17. Donaldson MS, Capron AM, eds. Patient Outcomes Research Teams: Managing Conflict of Interest. Washington, D.C.: National Academy Pr; 1991.

18. American College of Physicians Ethics Manual. Part II: Research, other ethical issues. Recommended reading. Ad Hoc Committee on Medical Ethics, American College of Physicians. Ann Intern Med. 1984; 101:263-74.

19. Institute of Medicine. The Responsible Conduct of Research in the Health Sciences: Report of a Study by a Committee on the Responsible Conduct of Research, Institute of Medicine, Division of Health Sciences Policy. Washington, D.C.: National Academy Pr; 1989.

20. Korenman SG, Shipp AC. Teaching the Responsible Conduct of Research through a Case Study Approach: A Handbook for Instructors. Washington, D.C.: Association of American Medical Colleges; 1994; 191-5.

21. Association of Academic Health Centers. Conflicts of Interest in Institutional Decision-Making: Task Force on Science Policy. Washington, D.C.: Association of Academic Health Centers; 1994.

22. AFCR guidelines for the responsible conduct of research. Clin Res. 1989; 37:510-1.

23. Harvard guidelines for investigators in scientific research. Clin Res. 1989; 37:192-3.

24. Donaldson MS, Capron AM. Patient Outcome Research Teams: Managing Conflicts of Interest. Washington, D.C.: National Academy Pr; 1991.

25. Hillman AL, Eisenberg JM, Pauly MV, Bloom BS, Glick H, Kinosian B, et al. Avoiding bias in the conduct and reporting of cost-effectiveness research sponsored by pharmaceutical companies. N Engl J Med. 1991; 324:1362-5.

26. Drummond MF. Economic evaluation of pharmaceuticals: science or marketing? Pharmacoeconomics. 1992; 1:8-13.

27. Freeman R. Health economics and strategic planning in pharmaceutical companies. Journal of Research in Pharmaceutical Economics. 1992; (In press).

28. The Zitter Group. The HMO Outcomes Study. San Francisco: Center for Outcomes Information; 1991.

29. Asch D. Opportunity and motive: conflicts of interest in industry-sponsored research. LDI Health Policy and Research Quarterly. 1981; 1:3.

30. Kassirer JP, Angell M. The journal's policy on cost-effectiveness analyses (Editorial). N Engl J Med. 1994; 331:669-70.