The levels of under-reporting noted by Jollis and associates [1] impressively show the severe limitation of claims data in defining prognostic risk factors of patients with ischemic heart disease. More than half the patients with specific conditions were not identified as such by the claims data.

In a recent coding audit of claims data collected by the State of California [2], we also found prognostic risk factors to be seriously under-reported. Applying the risk-adjustment model used by the Health Care Finance Administration for its annual hospital mortality release to the audited data, coding discrepancies led to a substantial, systematic measurement error that resulted in biased comparisons of the hospitals' risk-adjusted death rates. Jollis and associates speculated that some coding errors could reflect financial incentives of the DRG reimbursement system. Our study found a strong coding bias such that secondary diagnoses that could have increased DRG reimbursement were much more likely to be reported than were other conditions.

These troubling findings should not be surprising given previous research on the quality of claims data [3]. However, the frequency of coding errors and their systematic association with specific providers and outcomes raise questions about the suitability of these data for providing consumers with comparative provider-specific reports on outcomes [4]. Meanwhile, the use of claims data for such purposes continues; for example, California and Florida have recently issued hospital-specific data on adverse outcome rates derived from statewide claims files.

As Dr. Dans [5] notes, claims data are attractive to researchers (and policymakers) because of the low cost of obtaining huge numbers of cases. However, the true economic cost of claims data is not small if all the medical-records, financial, and data-processing personnel involved are considered. Given the widespread use of large databases for outcomes research and profiling, it may be time to consider a fundamental redesign of the data collection process. Such a redesign should incorporate the principles of data quality control used in clinical research and should include explicit definition of variables, standardized training, and constant monitoring. Central to this goal would be redefinition of the purpose of the data and its detachment from reimbursement incentives.