Direct Observation of Medical Trainees' Clinical Skills

Summaries for Patients are a service provided by Annals to help patients better understand the complicated and often mystifying language of modern medicine.

Summaries for Patients are presented for informational purposes only. These summaries are not a substitute for advice from your own medical provider. If you have questions about this material, or need medical advice about your own health or situation, please contact your physician. The summaries may be reproduced for not-for-profit educational purposes only. Any other uses must be approved by the American College of Physicians.

The summary below is from the full report titled "Effects of Training in Direct Observation of Medical Residents' Clinical Competence. A Randomized Trial." It is in the 1 June 2004 issue of Annals of Internal Medicine (volume 140, pages 874-881). The authors are E.S. Holmboe, R.E. Hawkins, and S.J. Huot.

What is the problem and what is known about it so far?

Doctors who teach medical students and residents often also evaluate them. Skills that they assess include interviewing and examining patients and discussing treatment options (counseling). There are multiple ways to evaluate trainees' skills. Many experts recommend watching the trainees during interactions with patients to assess skills, a method called direct observation of competence (DOC). Few faculty members receive training in DOC, and some may feel uneasy using it. Moreover, faculty teachers need feasible ways to learn DOC and need to know whether DOC training will improve their ability to assess trainees.

Why did the researchers do this particular study?

To compare comfort and skills in trainee evaluation between faculty teachers attending and not attending a 4-day development course that included training in DOC.

Who was studied?

40 faculty doctors from 16 internal medicine training programs.

How was the study done?

Programs were randomly assigned to have their faculty participate in DOC training or not. Faculty from both groups attended a meeting where they watched 9 videotapes of residents interacting with patients. Residents and patients in the videotapes used scripts. All videotapes showed residents making some errors in history taking, physical examination, or counseling. Some videotapes showed residents making many errors, while others showed residents making only a few errors. After viewing each videotape, faculty noted the errors and rated resident competence. Next, faculty in both groups received a notebook that provided information about ways to evaluate trainees' clinical skills. Then the group assigned to DOC training attended a 4-day workshop while the other group returned to their workplaces. The workshop included lectures and 1 day devoted to DOC training, which involved videotaped evaluation exercises and practice sessions. Eight months later, 37 of the original 40 faculty members rated a second set of 9 videotapes.

What did the researchers find?

Both groups gave similar ratings and missed many of the residents' errors on the first set of videotapes. Faculty who received DOC training rated residents more stringently on the second set of videotapes than those not receiving training. Faculty rated DOC training very highly and said they would definitely recommend it to colleagues. They said that DOC training made them more comfortable with direct observation.

What were the limitations of the study?

The study was small. Faculty rated videotapes of residents and patients who followed scripts rather than actual or real-life interactions.

What are the implications of the study?

Training in DOC is a promising way to improve faculty's ability to assess the clinical skills of trainees.