Medicare prescription drug event data are essential to any evaluation of the Part D program; however, the data have other important uses as well. The U.S. Food and Drug Administration proposed including Part D data in a new sentinel system to improve drug safety monitoring (3), but this has not yet happened. The Medicare Payment Advisory Commission has repeatedly requested Part D claims data to support its congressionally mandated responsibility to report on the effect of the drug benefit, to no avail (4, 5). The Medicare prescription drug event data represent an essential resource for conducting comparative effectiveness studies of medications used by the nation's elderly (6, 7). Apparently, these studies will not occur anytime soon.

An unintended consequence of not having access to Medicare prescription drug event data from CMS is that private researchers have sought alternative sources of information to answer critical questions about the operation of the drug benefit. The article by Yin and colleagues (8) in this issue represents an innovative example. The authors obtained prescription claims data from the Walgreens (Deerfield, Illinois) pharmacy chain for a large sample of Medicare beneficiaries age 66 to 79 years and a control group of noneligible persons age 60 to 63 years. They tracked each person's claims history monthly from September 2004 to April 2007. Key milestones in this period were January 2006, the first month of the Part D benefit, and May 2006, the final month in which beneficiaries could enroll in Part D that year. The 3 intervals defined by these dates represent a pre–Part D period, a ramp-up post–Part D period, and a stable post–Part D period. By using generalized estimating equation models, the authors derived conditional differences in trends in drug use (pill-days) and out-of-pocket spending for the Medicare and control samples during the 3 intervals. These measures should reflect 2 goals of the Part D program: to improve access to medications and to reduce the personal effect of medication expense.

Yin and colleagues (8) report that Medicare Part D made statistically significant differences in drug use and out-of-pocket payments. During the ramp-up period, Medicare beneficiaries were estimated to have increased pill-days by 1.1% per month (0.75 pill-day) more than what would have been expected in the absence of Part D, and their out-of-pocket payments decreased by 8.8%, or $3.80 per month. The estimated Part D effect on the average Medicare beneficiary was higher during the stable period, with pill-days increasing by 5.9% (3.7 pill-days) and out-of-pocket payments decreasing by 13.1% ($5.20 per month).

These results make the Part D effect seem small given the multibillion dollar cost of the program, but the denominator for calculating average changes in pill-days and out-of-pocket payments includes the entire cohort of Medicare beneficiaries, whether enrolled in Part D or not. A subanalysis limited to Medicare Part D enrollees found greater effects. During the ramp-up period, Part D enrollees had 5.9% more pill-days and 13.4% lower out-of-pocket payments. During the stable period, enrollees were estimated to have 16.1% to 32.6% more pill-days and 5.9% to 20.4% lower out-of-pocket costs (a considerable amount of money for many elderly persons), depending on when they enrolled.

Yin and colleagues (8) are not the first to study the effect of Part D by using Walgreens data. A similar study by Lichtenberg and Sun (9) was recently published. Two papers analyzing essentially the same data provide readers a rare opportunity to see how sensitive research results are to different methodological choices. Lichtenberg and Sun (9) also used a pre–post, comparison-group design, but they compared all people age 65 years or older with everyone younger than age 65 years, and they modeled the Part D effect as a simple difference-in-difference (pre– to post–January 2006) rather than the time-partitioned panel design used by Yin and colleagues (8). Lichtenberg and Sun (9) reported that Part D increased drug use among elderly persons by about 12.8% and reduced out-of-pocket payments by 18.4%. These results suggest that Part D had a substantially larger population-wide effect than reported by Yin and colleagues (8). Who is right?

Both studies have limitations. Walgreens is a large chain but handles less than 15% of the Medicare market, which raises questions about the generalizability of the results. Second, neither study has a record of prescriptions filled at other pharmacies. Most Part D plans have pharmacy networks made up of preferred providers. Because persons eligible for Medicare and Medicaid (dual eligibles) and other low-income subsidy beneficiaries are permitted to change plans monthly, any research strategy that tracks prescription claims from a single company is bound to underestimate total use. Finally, because neither study had access to actual Part D enrollment data, they cannot track month-to-month changes in a person's eligibility for Part D. Therefore, they cannot distinguish among alternative explanations for months in which a person has no pharmacy claims, which could represent nonuse by an eligible Part D enrollee, loss of enrollment (perhaps because of failure to pay required premiums), or death. Each of these explanations has a different effect on estimates of average, per-person drug use, and out-of-pocket expenses.

Despite these shortcomings, both studies indicate that Part D has accomplished much of what it set out to do. Before Part D, about 75% of beneficiaries had some form of drug coverage. By the end of 2006, 90% had it (4), representing approximately 6.3 million newly enfranchised elderly and disabled beneficiaries who were able to increase their use of medications at a substantial reduction in out-of-pocket cost. The challenge for the next round of research is to establish whether better access to medications has actually improved the health and quality of life of these individuals. This will require a database linking Medicare administrative records (denominator data plus Part A and B claims) with Part D plan enrollment, benefit design, and prescription claims data. Such a database is essential if we want to move beyond the big-picture effect of Part D and address questions relating to the quality and appropriateness of medication regimens for beneficiaries with specific acute and chronic conditions.