Mr. Broce and Dr. Reyes state that it remains unclear how the surveyed physicians derived post-test probabilities. They wondered whether we provided the relevant equations and whether we recorded if participants calculated or guessed post-test probabilities. In fact, we did neither. Therefore, we could not determine how the physicians arrived at their post-test probabilities. However, our experience and research have shown us that most physicians do not formally calculate post-test probabilities; instead, they use quantitative information about a test's informativeness in an inexact way (1, 2). Along the same line, we developed the inexact numerical graphical format. The setting of our trial, a lecture hall at a continuous medical education conference, was not conducive to studying the physicians' cognitive processes. However, we would welcome the opportunity to read reports of any studies investigating physicians' cognitive processes when they are confronted with quantitative information about a test's informativeness.

Dr. Brotman argues that our study did not test if the likelihood ratio (equal to 1) was the superior measure of association at extreme combinations of sensitivities and specificities (for example, a sensitivity of 0.97 and a specificity of 0.03). His hypothesis might be correct, but these extreme values of sensitivity and specificity are rare in commonly encountered diagnostic situations. We decided to present vignettes of more common clinical scenarios. Nevertheless, vignettes 1 and 4, which most closely resembled the situation that Dr. Brotman had preferred (vignette 1 had a sensitivity of 0.93 and specificity of 0.45 [likelihood ratio, 1.7]; vignette 4 had a sensitivity of 0.40 and a specificity of 0.79 [likelihood ratio, 0.8]), yielded only negligible differences between the 2 numerical formats on post-test probability estimates.

We agree that relevant experts should be involved in designing survey instruments before they administer them. Therefore, we tested and revised our vignettes with the help of 21 internists. We cannot refute the idea that a more sophisticated development process might have resulted in better vignettes. We agree that the vignettes' test–retest reliability could, and perhaps should, have been evaluated before use. However, we are not sure if the validity of the vignettes can be measured by any other means than through the eyes of experienced clinicians.

We welcome the suggestions for further studies that factor in the additional methodologic aspects highlighted by Mr. Broce and Drs. Reyes and Brotman. Or, in the spirit of Karl Popper: Design carefully and aim to refute so that you may corroborate convincingly.

For anyone who is interested, a copy of our questionnaire is available from the corresponding author.