Policy and Program Analysis Using Administrative Databases

15 October 1997 | Volume 127 Issue 5 Part 2 | Pages 712-718

Administrative policies and programs play an important and growing role as determinants of the use of medical care. Although some policies and programs may be harmful or ineffectual, randomized, controlled trials or prospective evaluations are rarely done for political or logistic reasons. Most evaluations are retrospective and often use administrative databases. Major problems with such evaluations include poor data quality, lack of concurrent controls, inability to ascertain important study outcomes, and incomplete data on case mix. This article uses published evaluations to illustrate these problems and suggests strategies that can minimize their impact. Such strategies include thorough assessment of data quality, interrupted time-series or policy gradient analysis, restriction of studies to those clinical outcomes that reliably result in medical care, and use of data on medical encounters as surrogates for determining case mix. However, even when these strategies are used, adequate evaluation of the effects of many policies and programs may continue to be impossible. Prospective evaluations need to be used more frequently to ensure that changes are held to the same standard used for other therapeutic interventions.

Author and Article Information

From Vanderbilt University School of Medicine, Nashville, Tennessee. For the current author address, see end of text.
Note: This article is one of a series of articles comprising an Annals of Internal Medicine supplement entitled "Measuring Quality, Outcomes, and Cost of Care Using Large Databases: The Sixth Regenstrief Conference." To see a complete list of the articles included in this supplement, please view its Table of Contents.
Grant Support: In part by grant HS07768 from the Agency for Health Care Policy and Research.
Requests for Reprints: Wayne A. Ray, PhD, Department of Preventive Medicine, Vanderbilt University School of Medicine, A-1124 MCN, Nashville, TN 37232.