 |
 |
|
Rapid Responses to:
|
|
Electronic letters published:
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif){kind=icon}
Response to Herman et al
Response to Eddy et al
Transparency and Face Validity in Simulation Modeling – Implications for Diabetes Prevention Policy
|
|
||||||||||||||||||||
|
David M Eddy, MD PhD Kaiser Permanente, Leonard Schlessinger PhD, Richard Kahn PhD
Send rapid response to journal:
eddyaspen{at}yahoo.com David M Eddy, et al.
|
Response. We are pleased to answer the questions raised by Herman et al. Concerning glycemic progression, they are correct that we used UKPDS data to help model the progression of glucose. We see that as strength, not a weakness. The resulting model accurately simulates the progression of diabetes across the entire time horizon of our analysis -- from IFG and IGT (it independently predicted the rates in the DPP) through to clinical diabetes and for at least 15 years thereafter (e.g. the UKPDS). See the online appendix to our for details (1). Herman et al argue for a faster rate of glycemic progression. Specifically, they assume everyone progresses from ¡°diabetes onset¡± to ¡°clinical diabetes¡± in exactly 10 years (2), which implies a rate more than twice that seen in the UKPDS. We see several problems with this assumption, beyond the fact that it contradicts the UKPDS. They base it on two papers (3) that estimated that retinopathy first begins to appear in populations about 4-7 years before clinical diagnosis. These papers in turn cited a study that followed 30 people after onset of diabetes and noted that the first case of retinopathy appeared 5 years after onset (4). The 10-year assumption comes from adding 5 years (onset to retinopathy) and 4-7 years (retinopathy to clinical diagnosis). First, 5 years was the shortest time from diabetes onset to retinopathy. The study actually showed a wide variation in transit times, with only 8 of the 30 (27%) even developing retinopathy at times ranging from 59 to 169 months after onset. Second, 4-7 years is the time of the first case in a population. It appears this has been misunderstood to mean retinopathy appears 4-7 years before clinical diabetes in everyone. In fact, the time of occurrence of retinopathy varied widely and on average was 10 to 20 years after clinical diagnosis. At the very least, these papers contradict a fixed 10-year transit time. Herman et al point to a 13% 3-year rate of early retinopathy in people in the DPP after onset of diabetes, and suggest that the eye complications we calculated are too low. The rates we reported are for advanced retinopathy (EDTRS ¡Ý60) in people who at the start of the calculation do not have diabetes. For early retinopathy, we calculated an 8% 3-year rate in people with pre-diabetes, which is in line with a 13% 3- year rate in people after they have developed diabetes. We also note that our model matches quite closely the progression of retinopathy seen in the UKPDS and other trials (5). Concerning myocardial infarctions (MI), the 12% in our paper is the probability a person from an otherwise relatively low risk population (e.g. 68% female, average BP = 123/78) with IGT (not yet diabetes) will have (not necessarily die of) at least one clinically recognized MI (not including silent or repeat MIs) in the coming 30 years (not a lifetime) in a setting of appropriate management of other risk factors (not average care). The 50% to which Herman refers is the proportion of deaths caused by CVD (e.g. including strokes and PVD) in people with diabetes in a setting of average care over everyone¡¯s lifetime. Thus the 12% and 50% are not comparable. A better place to start would be the rate of CVD events actually seen in the DPP placebo group after 3 years, which is 7.3 per 1000 person years (7.3/1000py) (6). Over the same 3-year period the rate we calculated for CVD events was 8.4/1000py. We predict that the DPP¡¯s rate will gradually increase as people age and develop diabetes; after 30-years of follow-up we calculate a rate of 18.7/1000py. As for causes of death, for people with diabetes about 40% are due to ischemic heart disease (7). For the DPP population we calculated 28%, which is what we would expect considering the differences in populations, shorter durations of diabetes, and shorter time horizons. Successful validations of MI rates in 12 clinical trials, including UKPDS and a prospective prediction of CARDS are described elsewhere (1 5). We can correct some other misunderstandings. First, we did not assume that glycemic progression is a ¡°linear process¡±; in our model it varies from person to person, and is not even linear for any particular person. Second, we did not ¡°constrain¡± diabetes progression nor did we fix A1c at 6.6%. The physicians in our simulation followed ADA guidelines and treated people to a target of 7% (making the average less than 7%). From that point on, we assumed the degree of control would gradually deteriorate as seen in the intensive care group of the UKPDS. Herman et al made similar assumptions (2). Third, we used the same model that had been validated against 18 clinical trials (5), and those validations do apply to this analysis. Fourth, we have validated the model for analyzing prediabetes; we did a blinded validation against the DPP itself with a good match (1 5). Fifth, we have searched the medical literature again and still cannot find any validations of the model Herman et al used. Perhaps we are using the word ¡°validation¡± in different ways; we are talking about comparisons of the model¡¯s calculations against real events (8). Herman et al argue their model is transparent whereas ours is not. First, we question the transparency of their model. To try to determine the reasons their results differ from ours we tried to reproduce their results, but were not able to because of inconsistencies and gaps in their reports (2). This said, we agree that our model uses a higher level of mathematics and more advanced programming and computing methods, and is therefore understandable by fewer people and more difficult to replicate. In addition to undergoing peer review by journals like the Annals, we make a technical report available through our website (1). We also meet with groups that are seriously interested in collaborating with us, to describe the model in detail. We have already met with Herman¡¯s colleagues at the CDC, but we offer to meet again to make both models mutually transparent. Herman et al argue their model has face validity, implying that our does not. Readers have to judge this for themselves. For some examples from their paper (P) or technical report (TR) (2)relating to this correspondence, the model Herman et al use assumes that everyone progresses from onset of early diabetes to clinical diabetes in exactly 10 years (P, 328;TR, 87); that the prevalence of retinopathy at beginning of clinical diabetes is 0% (TR, 9); that A1c increases 0.07% per year for 10 years during early diabetes (P, 328) and then jumps to a rate of 0.2% per year during clinical diabetes (TR, 22); that the percent reduction in diabetes progression seen at 3 years in the DPP stays constant forever (P, 324); that everyone with microalbuminurea has hypertension (TR, 76); that costs are multiplicative, not additive (P, Table 2); that the annual probability of developing ESRD does not depend on how long someone has had diabetes or even clinical nephropathy (Table 7b). See their papers for more examples (2). See our appendix (1) for a summary of the two models. Herman's summary of our conclusions is too negative. Our analysis confirms that exercise and weight loss are very important in helping prevent or postpone diabetes and its complications. Where we disagree with Herman et al is about the cost effectiveness of the intensive lifestyle intervention used in the DPP. At approximately $650 per person per year this intervention will simply be too expensive for most health plans, insurers and government programs, and it cannot be expected to return savings to significantly offset the costs over any practical time horizon. Less expensive methods for achieving weight loss need to be found. Failure to recognize this will only retard our progress in preventing this disease. David M Eddy MD PhD Leonard Schlessinger PhD Richard Kahn PhD 1. Eddy DM, Schlessinger L, Kahn R, Clinical outcomes and cost- effectiveness of strategies for managing people at high risk for diabetes. Ann Int Med. 2005;143-251-264. Online appendix available at www.annals.org. Website address www.archimedesmodel.com 2. Herman WH, Hoerger TJ, Brandle M, Hicks K, Sorenson, S, Zhang P, Hamman RF, Ackerman RT, Engelgau MM, Ratner RE, for the Diabetes Prevention Program Research Group. The Cost-Effectiveness of Lifestyle Modification or Metformin in Preventing Type 2 Diabetes in Adults with Impaired Glucose Tolerance, Annals of Internal Medicine. 2005;142:323-332. Technical report available at www.annals.org. 3. Harris MI, Klein R, Welborn TA, Knuiman MW. Onset of NIDDM occurs at least 4-7 years before clinical diagnosis. Diabetes Care 1992;15:815- 819. Thompson TJ, Engelgau MM, Hagazy M, Ali MM, Sous ES, Badran A, Herman WH. The onset of NIDDM and its relationship to clinical diagnosis in Egyptian adults. Diabetic Medicine, 1996; 13:337-340 4. Jarrett RJ. Duration of non-insulin dependent diabetes and development of retinopathy: analysis of possible risk factors. Diabetic Medicine 1986;3:261-263 5. Eddy DM, Schlessinger L, Validation of the Archimedes diabetes model, Diabetes Care. 2003;26:3102-3110. 6. DPP Research Group. Impact of intensive lifestyle and metformin therapy on cardiovascular disease risk factors in the diabetes prevention program. Diabetes Care 2005, 28: 888-894. 7. Geiss LS, Herman WH, Smith PJ. Mortality in non-insulin-dependent diabetes. Chapter 11 in Diabetes in America 2nd ed. National Diabetes Data Group, National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases, NIH Publication No. 95-1468,1995., see Figure 11.5 8. ADA Consensus Panel. Guidelines for computer modeling of diabetes and its complications¡± Diabetes Care. 2004 Sept;27:2262-2265 Conflict of Interest:None declared |
||||||||||||||||||||
|
|
||||||||||||||||||||
|
Andrew J. Palmer, BSc, MB, BS. CORE - Center for Outcomes Research, Stéphane Roze, William J. Valentine
Send rapid response to journal:
ap{at}thecenter.ch Andrew J. Palmer, et al.
|
Eddy et al (1) attempted to
assess the cost-effectiveness of type 2 diabetes prevention. The authors
projected 13.47% of a population with mean age 51 years and impaired glucose
tolerance (IGT) would die from coronary heart disease, stroke and renal
disease. They also report total life years (LY) over 30 years of 24.032 years/patient.
T We made some simple calculations
to put Eddy et al’s results into
perspective. Using age and gender-specific all-cause death rates for the
general US population (2), 51% of a
population age 51 years would die after 30 years (life years: 24.716). An IGT population with relative
risk of mortality of ~1.37 versus the normoglycemic population (3), and assuming no
progression to diabetes, mortality would increase to 56% (LY: 24.104). If we
assume Diabetes Prevention Program (DPP) control arm progression rates to
diabetes, and the associated 1.76-2.26-fold increased mortality (3), 30-year
mortality would be 75% (LY: 21.880). Based on these calculations, Eddy et al seem to underestimation total
mortality and overestimate life years, making further judgments on the
cost-effectiveness of DPP interventions based on these projections difficult. W Our group has published results of
cost-effectiveness analyses of the DPP, and have projected favourable
cost-effectiveness ratios/cost savings for intensive lifestyle changes or
metformin versus control in all country-specific settings analysed (3). If we plug US costs of intensive
lifestyle changes and metformin reported by Herman et al (6) into our model,
using a 30 year time horizon and assuming a duration of effect of the DPP
interventions lasting until death or development of type 2 diabetes, and brand
name (Glucophage) costs for metformin, we projected the following results:
*= Discounted at 3% annually We did not calculate
quality-adjusted life expectancy, but our costs/life year gained results fall
closer to Herman’s results than to those projected by Eddy, and suggest that in
a US setting, either intensive lifestyle changes or metformin would be
considered attractive from a cost-effectiveness viewpoint. REFERENCES (1) Eddy
DM, Schlessinger L, Kahn R. Clinical outcomes and cost-effectiveness of
strategies for managing people at high risk for diabetes. Ann Intern Med 2005;
143(4):251-264. (2) World
Health Organization. Life tables for 191 countries. World mortality in 2000. http://www3.who.int/whosis/life_tables/life_tables.cfm?path=evidence,life_tables&language=english. 2002. (3) Palmer
AJ, Roze S, Valentine WJ, Spinas GA, Shaw JE, Zimmet PZ. Intensive lifestyle
changes or metformin in patients with impaired glucose tolerance: modeling the
long-term health economic implications of the diabetes prevention program in
Australia, France, Germany, Switzerland, and the United Kingdom. Clin Ther
2004; 26(2):304-321. (4) Saydah
SH, Loria CM, Eberhardt MS, Brancati FL. Subclinical states of glucose
intolerance and risk of death in the U.S. Diabetes Care 2001; 24(3):447-453. (5) Colhoun
HM, Betteridge DJ, Durrington PN, Hitman GA, Neil HA, Livingstone SJ et al.
Primary prevention of cardiovascular disease with atorvastatin in type 2
diabetes in the Collaborative Atorvastatin Diabetes Study (CARDS): multicentre
randomised placebo-controlled trial. Lancet 2004;
364(9435):685-696. (6) Herman
WH, Hoerger TJ, Brandle M, Hicks K, Sorensen S, Zhang P et al. The
cost-effectiveness of lifestyle modification or metformin in preventing type 2
diabetes in adults with impaired glucose tolerance. Ann Intern Med 2005;
142(5):323-332. Conflict of Interest:Andrew J. Palmer, Stéphane Roze and William J. Valentine are employees of CORE, Center for Outcomes Research, which has received unrestricted research grants from Merck-Santé SA, Lyon, France. |
||||||||||||||||||||
|
|
||||||||||||||||||||
|
William H. Herman, MD, MPH Diabetes Prevention Program Research Group, Thomas J. Hoerger, PhD, Michael Brandle, MD, MS, Katherine Hicks, MS,Stephen W. Sorensen, PhD,Ping Zhang, PhD,Richard F. Hamman, MD, DrPH,David G. Marrero, PhD,Ronald T. Ackermann, MD, MPH,Michael M. Engelgau, MD, MS,Robert E. Ratner, MD
Send rapid response to journal:
dppmail{at}biostat.bsc.gwu.edu William H. Herman, et al.
|
We are writing in response to the report by Eddy et al.1 We have several questions about their simulations, conclusions, and representation of our work. Unlike Eddy, we found a favorable cost-effectiveness ratio for the Diabetes Prevention Program (DPP) lifestyle intervention.2 Archimedes may or may not represent a quantum leap in diabetes modeling. For simulation models to aid decision making, they must be transparent and have face validity. By using complex differential equations fitted to empirical data, Archimedes can simulate an infinite number of physiologic processes. Unfortunately, the equations governing disease progression are not transparent. In contrast, Markov models can simulate only a finite number of health states. Yet, as demonstrated in the appendix to Eddy’s report, Markov models are transparent and invite critics to debate their validity. In this simulation, unlike previous simulations, Eddy constrains the progression of hyperglycemia to reflect the increase in average fasting plasma glucose (FPG) over the first four years of the DPP (approximately 2 mg/dl/yr) and over the first fourteen years of the United Kingdom Prospective Diabetes Study (UKPDS) (3 mg/dl/yr). Thus, to progress from an FPG of 107 mg/dl at the start of DPP to 201 mg/dl at UKPDS presentation would take over 30 years. Studies have clearly demonstrated that progression from normal to impaired glucose tolerance and type 2 diabetes is not a linear process.3,4 Instead, a threshold is reached and there is rapid decompensation and progression.5 In this simulation, the glycemic exposure associated with diabetes is also limited. Eddy states that in each of their simulation strategies, “persons with diabetes whose HbA1c level exceeded 7% were entered into an intensive diabetes treatment program designed to reduce their HbA1c levels to below 7% … and reduced HbA1c level to an average of 6.6%.” In the U.S. population with self- reported diabetes, the median HbA1c is 7.5% and 41% of people have HbA1c ≥ 8.0%.6 These inputs into Archimedes may explain why the progression of hyperglycemia appears inordinately slow and diabetes appears to be without important clinical sequelae. In the DPP Outcomes Study, 13% of participants progressing to diabetes had evidence of diabetic retinopathy on retinal photographs taken on average 3.1 years after diagnosis (Late- breaking abstract presented at 2005 ADA, San Diego, CA). Moreover, we know that cardiovascular disease is common in diabetes, survival is decreased, and >50% die of cardiovascular causes. How then, after 30 years of follow-up, at an average age of 81 years, and after 72% of the cohort had developed diabetes, can Archimedes predict a 3% cumulative incidence of retinopathy, a 12% cumulative incidence of myocardial infarction, and 87% survival? We believe that it is due to the fact that hyperglycemia and diabetes were constrained not to progress. If hyperglycemia and diabetes do not progress, the risk of complications will be low, and the benefits of delaying or preventing the development of diabetes will be small. It also explains why prevention is not cost- effective. If people with diabetes do not develop late and expensive complications, then no intervention to prevent diabetes can improve health outcomes or provide economic value! Finally, we take exception to Eddy’s misrepresentation of our model and characterization of our model as “not validated.” Our model has been extensively validated and published since 2002. Admittedly, it has been adapted and refined over time – but that is true of most models including Archimedes. Indeed, because Archimedes was updated since the publication of initial validation studies, and because Archimedes was not previously used to model prediabetes, one could argue that Archimedes itself was “not validated.” If we were to believe the results of Eddy’s simulations, we would conclude that diabetes is not the clinical and public health problem that we know it to be. Archimedes’ projections simply do not reflect physiology or clinical reality. Because too few complications are predicted, diabetes prevention adds little value. Our model is transparent and has face validity. We stand by our results and believe that our conclusions are valid – that health policy should promote diabetes prevention in high-risk individuals.2 REFERENCES 1 Eddy DM, Schlessinger L, Kahn R. Clinical outcomes and cost- effectiveness of strategies for managing people at high risk for diabetes. Ann Intern Med. 2005;143:251-64. 2 Herman WH, Hoerger TJ, Brandle M, Hicks K, Sorensen S, Zhang P, Hamman RF, Ackerman RT, Engelgau MM, Ratner RE for the Diabetes Prevention Program Research Group. The cost-effectiveness of lifestyle modification or metformin in preventing type 2 diabetes in adults with impaired glucose tolerance. Ann Intern Med. 2005;142:323-32. 3 Knowler WC, Pettitt DJ, Savage PJ, Bennett PH. Diabetes incidence in Pima Indians: Contributions of obesity and parental diabetes. Am J Epid. 1981;113:144-56. 4 Ferrannini E, Nannipieri M, Williams K, Gonzales C, Haffner SM, Stern MP. Mode of onset of type 2 diabetes from normal or impaired glucose tolerance. Diabetes. 2004;53:160-5. 5 Ward WK, Beard JC, Halter JB, Pfeifer MA, Porte D Jr. Pathophysiology of insulin secretion in non-insulin-dependent diabetes mellitus. Diabetes Care. 1984;7:491-502. 6 Saaddine JB, Engelgau MM, Beckles GL, Gregg EW, Thompson TJ, Narayan KMV. A Diabetes Report Card for the United States: Quality of Care in the 1990s. Ann Intern Med. 2002;136:565-74. Conflict of Interest:None declared |
||||||||||||||||||||
