Health services research develops methods and measures to assess the quality of care provided by health plans, hospitals, physician groups, and individual physicians. Recognizing the probabilistic nature of health outcomes, health services research seeks to separate systematic variation (that is, variation in quality) from random variation in outcomes. Because patients with different clinical and socioeconomic characteristics have different prognoses, health services research seeks to adjust observed health outcomes for differences in underlying medical risks so that comparisons of variations in quality are fair. If relations between particular patterns of care and high-quality outcomes can be identified, health services research will help to provide a scientific foundation for evidence-based medicine. Finally, health services research evaluates the effects of different organizational, provider payment, and health insurance structures on patients' ability to receive quality care at reasonable cost and on an equitable basis.

To achieve these goals, health services researchers statistically analyze large databases that contain substantial amounts of personal information often obtained from various sources. Large databases are essential for separating systematic variation from random variation. Detailed personal information, both clinical and socioeconomic, is critical for adjusting for differences in underlying risks.

For those who work with such databases, the proposal by government [5] and medical [6] authorities to establish a health information infrastructure is of more than passing interest. We define the health information infrastructure as the framework that undergirds the electronic collection, storage, use, and transmission of information supporting the essential functions of the health care system [7]. This infrastructure could make health services research less expensive, more reliable, and more generalizable [8]. The availability of high-quality health information furthers many other social goods by supporting clinical decision making, improving quality, reducing cost, and promoting public health [9].

Substantial public benefits cannot effectively be achieved without systematic collection, use, and manipulation of health information. However, thoughtful observers resist disclosure of medical information and urge legislators to restrict access to the very data that are necessary for health services research [10]. One argument for limited access is founded on personal privacy and the confidential relationship between physician and patient. Institutions in the private sector, including the pharmaceutical, biotechnology, and managed care industries, also resist disclosure of information but often do so on grounds of proprietary interests.

The Health Insurance Portability and Accountability Act of 1996 [11] calls on the U.S. Congress to enact new privacy laws; if such privacy legislation is not enacted by August 1999, the U.S. Secretary of Health and Human Services is required to promulgate final privacy and security standards no more than 6 months later. The Secretary has already proposed standards for health information privacy [12]. Currently, health care researchers obtain access to databases, including Medicare and Medicaid databases, without specific informed consent. Under several proposed bills before Congress, however, access to these data would be restricted. As the government considers privacy regulation, how should it view the tradeoff between health services research and public benefit on one hand and privacy and proprietary interest in clinical data on the other?

The most important arguments for limiting access to patient-related data rest on privacy. Undoubtedly, reasonable levels of privacy are essential for the dignity of the patient and the development of a trusting therapeutic relationship. Standards for disclosure should vary with the extent to which data are personally identifiable. Release of aggregate data usually does not involve substantial privacy concerns because individual patients cannot realistically be identified. Aggregate data, of course, can harm ethnic, religious, or other social groups that may feel stigmatized by disclosure of information about the group's health status. The concerns expressed by Jewish groups about Tay-Sachs disease research or by African-American groups about sickle-cell anemia research illustrate the perceived harms attached to aggregate data. Still, aggregate data pose minimal privacy risks to individual persons and should be used in health services research whenever possible.

Despite the feasibility of using aggregate data to protect privacy, health services researchers claim (with justification) that identifiable, or linkable, data are often necessary for quality research. Personally identifiable information may be required to ensure accurate and complete records, avoid duplication of cases, and conduct follow-up studies. Data with personal identifiers are often needed to contact patients or their health care providers in order to conduct interviews, obtain medical releases, and collect supplemental medical and insurance claims information.

When personally identifiable data are used, informed consent is needed to respect the patient and maximize autonomy, although even the process of obtaining informed consent can create a sense of privacy invasion. Health services researchers often approach persons who have a particular disease to obtain their consent for the release of existing data or for participation in a follow-up study. Patients, understandably, may feel a loss of privacy because their names were disclosed to researchers. Furthermore, family members or neighbors may inadvertently learn of a patient's disease (for example, if a letter requesting informed consent is delivered to the wrong address). Consequently, the conflict between absolute privacy and the use of data for health services research can be eliminated only if the patient's health care provider personally requests informed consent and the patient grants it.

Despite the importance of explicit informed consent in protecting patient autonomy, requiring such consent before allowing any personal health data to be obtained would discourage and even halt much socially valuable research. The cost and effort of obtaining previous approval for large-scale statistical analyses that use data from tens of thousands of patients would be burdensome. Requiring previous consent to use existing data for health services research could also bias samples for statistical analyses. For example, a recent study reported that only 31% of persons responded to a mailed request to provide informed consent for the use of existing medical records data in an epidemiologic study of seizures; of those who responded, only 44% gave consent, a statistically meaningless sample [13]. Persons who withhold consent for use of their data are unlikely to be statistically similar to those who give consent. Inability to measure and adjust for potential self-selection biases in study design could seriously undermine the validity of health services research findings.

The conflict between a patient's desire for privacy and a researcher's need for data creates a social and ethical dilemma. On one hand, the use of aggregate data (or, if the data are identifiable, informed consent) is desirable in health services research. On the other hand, the communal good arising from health services research is greatly diminished when aggregation or consent would affect the reliability and quality of the research. There are no easy answers, but we should, at least, not assume that individual interests in privacy always trump collective interests in research.

It would be reasonable to permit expanded access to health information when researchers observe careful ethical standards: 1) Identifiable data should be collected only when necessary for research, 2) data should be collected and used strictly for scientific assessment of the health care system and other essential public health purposes, 3) researchers should store data securely so that only a few authorized users are permitted access to the database, and 4) secondary disclosures of personally identifiable data (for example, to employers, insurers, or commercial marketers) should be prohibited without the patient's informed consent. Researchers who violate patient privacy should be severely penalized. Access to personally identifiable health data without consent should also require impartial, outside scientific and ethical review that weighs the public benefits of the research, the measures taken to protect the confidentiality of identifiable data, and the potential harm to the individual patient that could result from disclosure.

Perhaps what the public desires is not absolute privacy. We all forgo some level of individual privacy in anticipation of the collective benefits of better health care and public health, and privacy that substantially impedes essential research may not be worth the social costs. The public may want reasonable assurances that when personal information is collected, researchers will treat that information with respect, store it in an orderly and secure manner, and use it only for important health purposes.

The pharmaceutical, biotechnology, and managed care industries frequently withhold research access to health information for a different reason-not necessarily to safeguard patient privacy but to protect their own proprietary interests. These industries are concerned that the data they reveal may give their competitors an advantage or that those data may deter employers or consumers from using their services. These claims have some merit. Health care industry research data can be used selectively and improperly to disparage individual firms or gain commercial advantage; society should take care not to unreasonably interfere with efficiency and some degree of profitability in the private sector.

Still, limiting research access to industry-generated health care data poses some serious concerns. The legitimacy of claims to proprietary interest is not independently scrutinized; commercial companies may simply be overprotective and secretive. Managed care organizations, for example, may resist disclosure for reasons that are socially unacceptable, such as to mask poor performance; arguably, consumers and government regulators have a right to objective evaluations of managed care plans. Even if proprietary claims are legitimate, they do not necessarily override the social importance of scientific research. As the health care system moves inexorably from fee-for-service practice to managed care systems and from nonprofit to for-profit health care plans, researchers may have less access to essential data because of proprietary justifications. This undermines the government's ability to monitor services that are necessary for the health of communities (services such as rates of childhood immunization, mammography, and colorectal screening). Proprietary justifications also impede valuable research evaluations of access, quality, and cost of the health care system. A reasonable balance between proprietary interests and access to research information is as necessary as the desired balance between the need for individual privacy and the value of research information.

Aside from the importance of evaluating specific managed care plans, other critical choices must be made about the social desirability of the managed care approach to health care. Health services research has shown that financial incentives offered by managed care, such as capitation, can result in patients receiving fewer services, usually at less cost, than do patients covered by traditional indemnity insurance plans [14, 15]. Do these differences affect patients' health? How well do managed care providers distinguish between necessary and unnecessary services when constraining use of resources? If costs are reduced, are the concomitant reductions in quality worth it? How are cost savings and quality reductions distributed among different social classes? These are ultimately social and political questions, and by making the appropriate data available, health services research can help legislators make wiser policy choices.

As a society, we value both privacy and the communal good that comes from collection of information to improve the health care and public health systems. We like to think that we can have it both ways, but we cannot: We will have to forgo some level of privacy to benefit from health services research. This diminution in privacy is not trivial, but the need to measure reasonable losses of privacy against collective expectations of considerable health benefit is inescapable. As Congress contemplates new privacy laws, it is important to consider how much would be lost by unreasonably restricting the collection of health information. The losses may be considerable.