Messori and colleagues are correct that, as editors, we must be wary of results from non-validated, home-grown statistical programs. However, the possibility of calculation errors is not the only reason for this caution. Ironically, the authors of the letter may have provided an excellent example of the other reason.

Hommes and associates did not indicate what kind of software they used. This may have been an editorial oversight, since we usually ask that all software be cited or its source described. However, their published results are confirmed exactly by a reanalysis using two commercial programs (Egret [1] for logistic regression and StatXact [2] for Mantel-Haenszel analyses). The disparity noted by Messori and associates probably occurred because they used the Hauck [3] estimator of the log odds ratio variance, which has been superseded by the Robins-Breslow-Greenland [4] estimate. The Hauck formula is not used in commercial programs because it gives inaccurate results when there are small cell sizes and few tables, both of which occurred here. Thus, while Messori and colleagues are correct to warn that nonvalidated software can have computational errors, of equal concern is that such programs often do not use formulae chosen by experts.

Finally, Messori and colleagues' assumption that various "well-accepted" methods should produce similar outcomes is not always correct. Even with commercial software, different assumptions underlying various analytical approaches can produce quite different results. Statistical expertise is necessary to understand why. This raises the wider issue of the problems posed by having powerful statistical techniques widely available on personal computers to researchers not familiar with their nuances or limitations. But that is a subject for a future commentary.