Dr. Steensma is concerned about the carryover effect of reusing questions on the IM-ITE after a gap of 2 or more years. The operational practice of reusing items from previous examinations is similar for virtually all national examinations that use multiple-choice questions. The IM-ITE, however, is unique in that most examinees take the test in 3 consecutive years. The practice of reusing items provides the examination with a core group of questions that have performed well on psychometric evaluations in the past and have been demonstrated to be effective in distinguishing high scorers from low scorers. It also enables psychometricians to equate the scores of the examination from one year to the next. The previously used items that we include on the IM-ITE are from several different administrations and not from one single examination, with a gap of at least 1 year between use. Through internal audits, we have found no statistically significant difference in performance on new versus previously used items (Subhiyah RG, McGrenra CC. Comparison of performance of ACP/ASIM ITE candidates on new and used items. 1999, 2000, 2001. Unpublished data). Similar comparisons have been made for other programs, with similar results (Subhiyah RG, Hess B. Effect of item exposure on a take-home recertification examination. 2002. Unpublished data). Furthermore, in our paper, we noted a consistent increase in examination scores between the PGY1 and PGY2 years, when the examinations were totally different, and the PGY2 and PGY3 years, when some questions may have been repeated. We did not see a marked increase in scores in the PGY3 examination, which might have been anticipated if residents had actually remembered questions that had been asked previously.

Dr. Nidiry is concerned about our finding that international medical school graduates have outscored U.S. medical school graduates on the last seven IM-ITEs. Before 1995, unbeknownst to the IM-ITE committee, the examination had been "speeded" for those less practiced in the English language. In 1995, the time allocated for the IM-ITE was lengthened from 6 hours to 7 hours. In that year, the scores of international graduates in all 3 training years increased substantially and exceeded those of U.S. graduates. International graduates have scored consistently higher on all subsequent examinations with the longer time for answering questions. These data suggest that graduates from international medical schools might be better trained than U.S. graduates or that they may have benefited from extra clinical experiences before entering a U.S. residency program. It is also possible, as Drs. Dibner, Andrieni, and Smith suggest, that international medical school graduates may have been selected specifically because of their success on a similar type of examination when they applied for residency training. Our data simply point out that international medical graduates were better able to demonstrate their medical knowledge when the time limit for the examination was increased sufficiently to enable them to complete the test.

The IM-ITE does not compare the performances of U.S. graduates in internal medicine training programs with those of U.S. graduates in other specialties in which the content differs. Thus, our study provides no insight into Dr. Nidiry's idea that graduates of U.S. medical schools in internal medicine residency programs are less knowledgeable than those entering other residency programs.