The question of how to generalize data from randomized trials for use in real-world decisions is timely and important. Many decision makers desire credible information about the clinical and economic effects of new interventions. In response to this, Section 114 of the recently passed Food and Drug Administration Modernization Act of 1997 regulates the dissemination of pharmacoeconomic information that is "competent and reliable" [1].

Representatives of the Food and Drug Administration, public interest groups, the legislative committee that drafted the bill, managed care organizations, the pharmaceutical industry, and academia recently convened in Washington, D.C., at a special briefing of the act sponsored by the International Society of Pharmacoeconomic and Outcomes Research (ISPOR). The sentiment of many speakers was that randomized trials, although providing the greatest assurance of internal validity, fall short of providing the data needed for economic analyses to be useful in the real world.

Sacristan and colleagues comment on one key limitation of randomized trials: the challenge of generalizing results from a highly selected trial population to the broader base of patients cared for in managed care plans. They offer a "randomized databases" solution that warrants critical appraisal. But there are other concerns, raised at the ISPOR briefing, about basing economic inferences on only the results of randomized trials. First, the variability in economic end points-both costs and quality of life-is often greater than the variability in clinical end points. The sample size of trials may be insufficient to detect significant differences in economic end points. By contrast, Bayesian approaches permit estimation of sample sizes that are economically sensible, considering the costs of data collection and the value of new economic information [2]. Second, the medical resources actually used in trials are often driven by the research protocol in response to regulatory concerns, which can limit the generalizability of the findings when applied to real-world situations. Third, the costs of conducting long-term trials can be substantial, so that the only practical alternative is to measure in the trial the effect of a new intervention on a surrogate end point, which then may be linked to other observational data to estimate the potential long-term consequences [3]. The ever-increasing emphasis on health maintenance instead of disease treatment further underscores the need for methods with which to estimate potential long-term consequences. Most technology must be adopted before all long-term consequences can be established with certainty.

Controversy remains about how best to overcome the limitations of randomized trials. Different constituencies, despite their varied agendas, have expressed concern that existing methods of randomized trials do not provide the quality and timeliness of evidence that they and the public need for making real-world decisions. It is encouraging, however, that these groups are striving to work together to develop competent and reliable standards, perhaps considering new solutions, for estimating the clinical and economic consequences of new therapies.