RSS Feeds
-
The issues raised do not invalidate study conclusions
Submit responseBerger raises two criticisms of our study. First, he claims that permuted block randomization with fixed block size of five was inappropriate as it could lead to prediction of future allocations. While such prediction is a valid theoretical concern, it implies an unlikely situation where study personnel manipulate the randomization scheme at every step. Block size was neither included in the protocol nor made known to study personnel during the conduct of the study. Allocations were implemented with a voice response system which was transparent to study personnel. Thus, prediction of allocations was mitigated while the block size ensured balance among intervention groups within each center, an important consideration for an interventional study. As evidence of the inadequacy of our randomization scheme, Berger notes a “maldistribution” of race, but in doing so, he has chosen a single imbalanced characteristic out of a large number of possible characteristics. We formally compared baseline characteristics among intervention groups, and found no significant differences. Specifically with respect to race, the majority of subjects were Caucasian and the proportion of Caucasians was not significantly different among intervention groups. The fact that the relatively small number of blacks was unevenly distributed was likely nothing more than an accident of randomization that has little, if any, effect on the results. Berger goes on to suggest that the maximal procedure, which enforces balance on known covariates such as race, should have been chosen over permuted blocks; however, there is no definitive opinion within the wider statistical community on the question of whether covariates should be taken into account in trial design (1).
The second criticism refers to the dropout rate and the lack of a pure intention-to-treat analysis. Drop-out is a common occurrence in weight loss studies, and the rates in this trial which ranged from 26-34% across intervention groups were moderate in comparison to other studies in the literature (2). Furthermore, as the most common categories of drop-out were loss-to-follow-up and consent withdrawal, we are uncertain whether efforts to obtain 6-month weight measurements from dropouts would have been successful. Berger states that we failed to include the dropouts in the analysis, but this is not the case. In the primary longitudinal analysis, drop-outs were included to the extent to which their data was available. In this regard, the analysis satisfies the intent-to-treat principle and preserves the randomization for the purpose of statistical inference. We also conducted sensitivity analyses including a worst-case scenario that assumes that every drop-out returned to their baseline weight by 6-months. While the magnitude of weight loss observed was decreased in this scenario, the relative pattern among the treatment groups was maintained. Berger also questions the assumptions of the primary analysis. We do not feel that an assumption of normality is unrealistic for body weight data in a trial of this size. Furthermore, the mixed model repeated measures approach has been recommended by experts (3, 4) as a means of dealing with missing data in longitudinal studies.
References
1. Rosenberger WF, Sverdlov O (2008). Handling covariates in the design of clinical trials. Statistical Science, 23:404-419.
2. Pi-Sunyer FX, Aronne LJ, Heshmati HM, Devin J, Rosenstock J (2006). Effect of Rimonabant, a cannabinoid-1 receptor blocker, on weight and cardiometabolic risk factors in overweight or obese patients. JAMA, 295:761-775.
3. Gadbury GL, Coffey CS, Allison DB (2003). Modern statistical methods for handling missing repeated measurements in obesity trial data: beyond LOCF. Obesity Reviews, 4:175-184.
4. Mallinckrodt CH, Lane PW, Schnell D, Peng Y, Mancuso JP (2008). Recommendations for the primary analysis of continuous endpoints in longitudinal clinical trials. Drug Information Journal, 42:303-319.
Conflict of Interest:
None declared
-
A Fatally Flawed Trial
Submit responseEditor: When the camera is sufficiently out of focus so as to preclude the possibility of identifying what or who is in the photo, one can imagine that there would be no shortage of expert consultants who, for a fee, will help you figure it out. After all, since you do not know what is really there, you can never prove them wrong. Likewise, when a clinical trial makes one methodological error after another, it is fun to pretend that we can still use it to draw conclusions. But are these conclusions justified? Are obese patients really to trust the results of a trial (1) that randomized inappropriately, allowed the study cohort to be decimated by drop outs, did not account for these myriad numbers of drop outs in the “modified” intent to treat analysis, and based the analysis on distributional and other assumptions that could not possibly be true? If you will pardon the pun, this is just a bit hard to swallow.
In an unmasked trial, such as this one, the worst form of randomization is permuted blocks, especially with a small, fixed block size (2,3), such as the fixed block size of five that was used in this trial. With five treatments, this means that each block allocated each treatment only once; hence, five is the smallest possible block size, and leads to the most prediction of future allocations. Allocation concealment then becomes impossible. Notice the maldistribution of race among the treatment groups. A more appropriate method of randomization, perhaps the maximal procedure (2, 3), should have been used instead of permuted blocks. The large numbers of drop outs may have been unavoidable, but failure to include them in the analysis is inexcusable. We all know the merits of the intent to treat approach, and the rationale for using it. There is a flip side to this; if using a method (such as intent to treat) is good, then surely not using it must be bad. That corollary certainly applies in this case, as the exclusion of randomized patients opens the analysis to all kinds of biases. This is especially true when the drop outs are large in number and uneven across treatment groups, as they are here. Moreover, the primary analysis was a repeated measures linear model which piles assumption upon assumption, not the least of which is normality. When the analysis drowns out the data, rather than reflecting the data, it cannot be taken seriously. The flawed randomization (permuted blocks with a small fixed block size in an unmasked trial), the large and uneven number of drop outs, the failure to include all randomized patients in the analysis, and the use of an overly optimistic analysis that in no way reflects the limitations of the data as collected all conspire to cast intense doubt on all conclusions deriving from this study. About all that can be concluded with certainty is that we need better studies.
References:
(1) Digenio AG, Mancuso JP, Gerber RA, Dvorak RV. Comparison of Methods for Delivering a Lifestyle Modification Program for Obese Patients. Annals of Internal Medicine 2009; 150:255-262.
(2) Berger, VW, Ivanova, A, Deloria-Knoll, M (2003). “Minimizing Predictability while Retaining Balance through the Use of Less Restrictive Randomization Procedures”, Statistics in Medicine 22, 19, 3017-3028.
(3) Berger, VW (2005). “Selection Bias and Covariate Imbalances in Randomized Clinical Trials”, John Wiley & Sons, Chichester.
Conflict of Interest:
None declared