Garibaldi and colleagues [1] discuss the usefulness and reliability of the In-Training Examination in Internal Medicine (ITE-IM). They note how residents and program directors can use scores to evaluate individual and group performances. We found that the ITE-IM has a strong positive predictive value in identifying residents who pass the certifying examination in internal medicine of the American Board of Internal Medicine (ABIM).

Since 1988, all residents have taken the ITE-IM during their second year of residency and, since 1993, during both their second and third years. We analyzed the relation between the residents' performance on the ITE-IM during their second year and their ABIM examination passing status on their first attempt. Data were analyzed using chi-square analysis and the Fisher exact test.

Overall, 104 of 111 residents graduating between 1989 and 1993 completed both the ITE-IM and the ABIM examination. We used the 25th percentile as a cut point to determine the probability of passing the certifying examination.

The sensitivity of predicting whether a resident would pass the ABIM examination was 90% Table 1, and the specificity was 60% (P > 0.001). Of the eight residents who failed the ABIM examination despite an ITE-IM score in the 25th percentile or greater, the mean ITE-IM percentile was 45.6% compared with 60.2% for those who passed the ABIM examination.

An ITE-IM score in the 25th percentile or greater was an absolute predictor of passing the ABIM for third-year residents who took the ITE-IM in 1993. Of these 24 residents, all 18 who scored above this cut point passed the ABIM examination, and all 6 who scored below failed it (P < 0.001).

The use of percentile grades rather than the percentage correct score provides a standardized measurement for comparing residents with their peers and with residents from previous years. The program director can use the ITE-IM to determine a resident's chance of passing the ABIM examination and then structure a learning program targeted to helping residents become certified on their first attempt. Because scores are further divided by subspecialty, residents can focus on specific areas in which their knowledge needs strengthening. These results help to validate the interpretation of the usefulness and reliability of the ITE-IM.