McDonald and colleagues raise important points. First, the weight we gave to the unit of randomization was based on concerns about contamination. This was judged to be especially important in evaluating computerized decision support systems, particularly early in their development, and the effects to date have been small. Thus, we did not wish to bias our review against such systems by considering those that had been tested with randomization within groups to be on the same footing as those with randomization in separate, independent groups. It is true, however, that, in a given circumstance, the most appropriate approach to randomization may not be the theoretically preferable one and that our decision reduces the precision of testing for reductions in contamination.

Second, we did not require that studies assess effects on patient outcomes but accorded equal status in the scoring system to measures of clinician performance and patient outcomes. As noted in the Methods section, studies could receive full marks for measuring either, if they met the criteria stated in the Appendix. Unfortunately, we used the term "outcome" ambiguously in the scoring system, indicating the effect of the computer-based decision support system on either clinician performance or patient events. We are grateful for the opportunity to clarify this point. We do agree, however, that at least intermediate patient outcomes should be measured in the final stages of testing of computer-based decision support systems whenever feasible.