Educational programs for practicing physicians that rely on seminars or written materials are often unsuccessful in improving physicians' prevention practices [11]. However, these practices have increased as a result of skills-based training [11-13]. For example, simulated patient instructors (Instructors), persons trained to role-play specific types of patients, coupled with feedback increased the clinical competence of physicians-in-training [14]. A study of the effect of mailed educational materials, including printed materials, audiocassettes, or videocassettes, on increasing the clinical competence of office-based physicians regarding AIDS showed that no effect occurred [15]. An office-based intervention using mailed educational materials and Instructors has not been evaluated for increasing physicians' prevention practices for sexually transmitted diseases and HIV infection.

In our previous study, we described rates of physician risk assessment and counseling practices regarding sexually transmitted diseases and HIV infection before intervention [5, 16, 17], the importance of preparation before simulated patient instruction [18], and the acceptability and feasibility of using simulated patients for instruction [19] and evaluation [20]. We report here the effect of interventions on increasing physicians' prevention practices regarding sexually transmitted diseases and HIV infection. We conducted a randomized controlled trial to determine if the prevention practices of office-based primary care physicians could be increased by mailed educational materials alone or by mailed educational materials combined with Instructor role-play and feedback.

Methods

Top Methods Results Discussion Author & Article Info References

Study Design

This project was approved by Georgetown University's Institutional Review Board. The study was a community-based, randomized controlled trial of educational interventions (Figure 1). The physician sample frame was the 1988 American Medical Association (AMA) mailing list (members and nonmembers) of all office-based internists, general and family practitioners, and obstetrician-gynecologists in the Washington, D.C., Metropolitan Statistical Area. To ensure manageable implementation of the educational interventions, the sample frame was randomly divided in half. The same study protocol was conducted with the second half of the sample 3 months after the first half.

View larger version (34K): [in this window] [in a new window]   Figure 1. Study design of a randomized controlled trial of educational interventions to improve physicians' prevention practices regarding sexually transmitted diseases and HIV infection. Educational materials (EM) were mailed an average of 4 months after the initial survey. Physicians were visited by simulated patient instructors an average of 6 months after the initial survey. The survey after the intervention was done an average of 9 months after the initial survey, and physicians were visited by simulated patient evaluators an average of 11 months after the initial survey.

Physicians were mailed letters from the Director of the National Institute of Allergy and Infectious Diseases to introduce the study and from the project directors to describe study procedures and the availability of continuing medical education credit for physicians completing the study. Physicians were excluded from the sample at the time of the telephone interview done before intervention if they could not be located; were deceased, retired, or in training; or reported no primary care practice. Of 1605 eligible physicians, 961 completed this interview, and 757 completed the interview after the intervention. Some physicians were lost to follow-up for the latter interview because they became ineligible (n = 93) or because they refused continued study participation (n = 111). When study participants (n = 757) were compared with nonparticipants (n = 848) regarding five AMA mailing list characteristics (sex, type of practice, primary care specialty, board certification in primary care specialty, and state of primary practice site), no significant differences were detected.

After the initial interview, study physicians were stratified by the Washington, D.C., Metropolitan Statistical Area region of practice (Washington, D.C.; Virginia; and Maryland) and specialty (internal medicine, family practice, general practice, and obstetrics-gynecology) and were randomly assigned to a study group on the basis of a table of random numbers. The study groups were as follows: Group I was the control group; group II received mailed educational materials; and group III received mailed educational materials plus an office visit from an Instructor. Follow-up rates of interviews after intervention were 79% in group I, 80% in group II, and 77% in group III.

Interventions

The educational materials were mailed to physicians approximately 4 months after the initial interview. The materials consisted of a binder with the following: 1) cover letters from the Director of the National Institute for Allergy and Infectious Diseases and from the project directors; 2) an audiotape modeling physician assessment of sexual risk and risk reduction counseling of various patients; 3) a monograph on "Risk Assessment and Counseling to Prevent STD/HIV Infection"; 4) sample patient brochures on sexually transmitted diseases, AIDS, and condoms; 5) U.S. Preventive Services Task Force recommendations for screening and treating sexually transmitted diseases; 6) a Washington, D.C., Metropolitan Statistical Area resource directory for persons with HIV infection and AIDS; and 7) a Centers for Disease Control and Prevention wall chart on diagnosing and treating sexually transmitted diseases.

Physicians were visited by Instructors during the 3 months after the educational materials were mailed. Instructors were young women trained to play the role of "Emma Smith." Because obstetrician-gynecologists were included among study physicians, female Instructors were chosen to standardize the simulated patient across specialties. Criteria for Instructors included having a bachelor's degree, acting experience, access to an automobile, and flexible work hours. Instructors received intensive training and achieved high reliability [18, 19]. Emma's office visits were scheduled after the physicians were informed about the purpose of the visit. Emma was a 28-year-old computer programmer who had recently moved to Washington, D.C., from a small Midwestern town. She reported having a recent onset of vaginal discharge that was later shown to be trichomoniasis. Emma's history, if fully elicited, included multiple sexual partners, previous sexually transmitted diseases, and vaginal and oral intercourse without condoms. The physician role-played assessing risks and counseling regarding sexually transmitted diseases and HIV infection with Emma for an average of approximately 15 minutes. A physical and laboratory findings card was used in place of a direct examination. After the role-playing, the Instructor provided the physician with an average of approximately 10 minutes of verbal feedback on the interaction from a patient's perspective, followed within 3 weeks by a mailed letter reiterating the major deficiencies observed in the physician's prevention care for sexually transmitted diseases and HIV infection. Immediately after the visit, the Instructors recorded observations on standardized assessment forms and returned them to the field supervisor.

Self-reported Practice

The effect of the interventions on physician practice was evaluated by both physician self-report and observed physician practice. After the intervention, physicians received a telephone interview that occurred an average of 9 months after the initial telephone interview, an average of 5 months after educational materials were mailed, and an average of 3 months after the Instructor visited physicians in group III. The interview after the intervention contained many of the same questions on physician prevention practices for sexually transmitted diseases and HIV infection that were contained in the initial survey.

All telephone survey questions were close-ended, with trained interviewers recording responses on precoded forms. Physician interviews were done by WESTAT, Inc., Rockville, Maryland, a telephone survey research center.

Physician questioning of patient risks for sexually transmitted diseases and HIV infection was measured by asking physicians how often on a 5-point scale (ranging from always to never) they ask new patients about various sexually transmitted diseases and HIV risk factors during initial general health examinations.

A female patient scenario was used to assess the frequency with which physicians counseled female patients to reduce the risk for sexually transmitted diseases and HIV infection. Physicians were asked how often (on a 5-point scale, always to never) they recommend various safer sex techniques to their female patients who are not married, have had more than one sexual partner within the last 3 months, have only recently used condoms, and have no signs or symptoms of sexually transmitted diseases, including HIV infection. Twenty-eight physicians could not answer the question; most nonresponders said they had no patients who fit the scenario.

During the interview after the intervention, physicians were asked to compare their current practice regarding sexual history-taking and risk reduction recommendations with their practice 6 months before.

Observed Practice

Because physician self-reported practices could be biased, a follow-up interview validation study was done with the second half of the physician sample. At the end of the interviews done after the intervention, consent was obtained from study physicians for an unannounced office visit by an evaluator pretending to be a patient—a simulated patient evaluator (Evaluator)—some time during the next 3 months. Physicians were reimbursed $60 for this visit. Whereas the physician knew when the Instructor visit was scheduled, the Evaluator visit was scheduled as a new patient with no announcement that the patient was a surrogate.

A few physicians in the first half of the sample served as subjects for pilot testing the Evaluator, and data from these pilot visits were not used in the analyses. Physicians in the second half of the sample were eligible for the Evaluator office visit and were asked to participate if they accepted new fee-for-service patients. Of the 355 physicians asked to consent to an Evaluator visit, 17% were ineligible because they did not accept new fee-for-service patients, 25% refused to consent, and 3% became ineligible for other reasons or could not be visited. Physicians visited by an Evaluator (n = 194) were equally distributed across the three study groups. No significant differences in physician or physician practice characteristics were noted across the three study groups or between the study physicians visited by an Evaluator and all study physicians (n = 757).

Evaluators were women trained to play the role of "Judy Green," a 31-year-old recent divorcee who was new to the area and concerned about becoming sexually active with a new boyfriend [20]. Evaluators received intensive training and achieved high reliability [17, 20]. Judy requested a consultation only and refused physical examination. Judy's history included previous sexually transmitted diseases, intercourse with partners she did not know well, and lack of condom use. Judy's visit was scheduled as a new patient visit. The Evaluators did not know the study group of the physician they visited. Sixteen percent of the physicians who received an Evaluator reported at the end of the visit that at some time during the visit they suspected the patient was an Evaluator.

Evaluators completed standardized forms about physicians' risk questioning and risk reduction counseling immediately after each physician visit. Ratings were dichotomous: The topics were either covered or not covered. Evaluators did not discuss the visit with physicians before completing their forms.

Analysis

Interview variable means were compared across study groups I, II, and III before intervention using a one-way analysis of variance [21]. Trends of means from before to after intervention across study groups I, II, and III were compared by a repeated analysis of variance interaction effect. We did a post hoc test for a one-way analysis of variance on the means after the intervention on each variable that had a significant repeated-measures group by phase (before or after intervention) interaction effect to interpret which groups differed after the intervention. Multiple pairwise comparisons were done using Tukey's procedures [21]. Differences in Evaluator measurements across study groups were examined using a chi-square test. When differences across groups were detected by using an overall chi-square test, multiple 2 x 2 contingency table chi-square comparisons were done to determine which groups differed.

Results

Top Methods Results Discussion Author & Article Info References

Success of Randomization

Comparisons of 23 physician characteristics across study groups indicated two differences: the number of hours of primary care provided per week and the percentage of patients aged 65 years or older (Table 1). Physician characteristics were also compared across groups in the Evaluator observation study; no statistical differences were indicated.

Intervention Exposure

In the interview after intervention, group III remembered receiving the educational materials more often than did group II (95% compared with 66%) (P < 0.001). Also, compared with group II, group III reported reviewing all or part of the monograph on sexually transmitted diseases and HIV infection prevention (89% compared with 53%; P < 0.001), all or part of the sample patient education brochures (84% compared with 48%; P < 0.001), and all or part of the audiotape that modeled physician sexual history-taking with different types of patients (68% compared with 33%; P < 0.001). In group III, 34% reported reviewing the materials before the Instructor visit, 22% reported reviewing the materials after the visit, and 38% reported reviewing the materials both before and after the visit. Comparisons included all intended and completed visits.

Change in Risk Questioning for Sexually Transmitted Diseases and HIV Infection

Before study interventions, study groups did not differ statistically in the frequency with which they asked new patients about eight topics related to risks for sexually transmitted diseases and HIV infection. After the intervention, group III reported questioning more new patients about anal and oral sexual practices and partners' intravenous drug histories than did group I (Table 2). In addition, group III reported questioning more patients about their sexual orientation after the intervention than did group II. Furthermore, more physicians in Group III reported an increase in their questioning of patients about their sexual histories than did physicians in both groups I and II (43.5% for group I compared with 55.7% for group II and 76.2% for group III) (P < 0.001). During visits, Evaluators observed that group III physicians asked about condom use and anal sex practices more often than did group I and about oral sex practices more often than did groups I and II (Table 3).

Change in Counseling for Preventing Sexually Transmitted Diseases and HIV Infection

Before study interventions, study groups were similar in the frequency with which they advised women at risk about five topics on preventing sexually transmitted diseases and HIV infection. After intervention, group III reported providing women more advice about using condoms for vaginal and oral intercourse and being monogamous than did groups I and II (Table 4). Also, after intervention, group III reported providing women more advice about using condoms for anal intercourse than did group I. Furthermore, more physicians in group III reported an increase in the number of specific recommendations they gave patients about reducing their sexual risks than did physicians in groups I and II (42.2% for group I compared with 52.6% for group II and 72.9% for group III) (P < 0.001). During their office visits, Evaluators observed that physicians in group III advised "Judy Green" about the importance of using condoms (but did not relate condom use to specific sexual acts) and provided educational materials on condoms more often than did physicians in groups I and II (Table 5).

Subgroup Analysis of Changes in Reported Practices before and after Intervention

We analyzed changes after intervention in physician-reported practices by study group for interactions with the physician characteristics presented in Table 1. No interactions were present between these characteristics and changes in reported practices after intervention.

Discussion

Top Methods Results Discussion Author & Article Info References

One of the national health objectives for the year 2000 is that 75% of primary care providers and 90% of providers in high-risk areas provide age-appropriate counseling on preventing sexually transmitted diseases and HIV infection [22]. Surveys of physicians in California [3], New York [6], Washington, D.C [5]., and Ohio [4], as well as national public surveys [2, 9], indicate that primary care physician practice is currently well below this goal. Ours is the first reported study providing evidence that office-based primary care physicians' prevention practices regarding sexually transmitted diseases and HIV infection can be increased. Study results indicate that the combination of educational materials, simulated patient instructor office visits for role-play, and feedback on clinical performance increases office-based physicians' risk assessment and risk reduction counseling regarding sexually transmitted diseases and HIV infection, thereby improving physicians' prevention practices.

Our study confirms previous reports that mass mailing educational materials alone does not increase the risk assessment and counseling practices of office-based physicians. One reason for this is that physicians often do not review the educational materials on AIDS that are mailed to them [15]. Only two thirds of the physicians who were mailed educational materials as their sole form of intervention recalled receiving them. About one third of the physicians who remembered receiving the materials thoroughly reviewed them. In contrast, nearly all physicians who received an Instructor visit after the educational materials were mailed recalled receiving and reviewing the materials. Almost two thirds of physicians in the study group who were visited by the Instructor reviewed the materials after the visit, further indicating that the Instructor motivated review of the materials. The educational materials undoubtedly enhanced the effectiveness of the Instructor in increasing the preventive care practices for sexually transmitted diseases and HIV infection [18]. It is unclear from our study if Instructors alone can increase physicians' risk assessment of and risk reduction practices for sexually transmitted diseases and HIV infection if educational materials are not concurrently provided. We recommend that relevant educational materials be used with Instructors when physicians are trained.

This study corroborates other studies indicating that skills-based strategies are more effective than information dissemination alone in changing actual practice [11, 13]. Study findings are based on physician self-reported practices and are corroborated by observed practices. Although most studies that have measured physicians' HIV prevention practices have relied on physician self-report [3-6, 8], these self-reports on providing behavioral counseling have been observed to be unreliable [23, 24]. In our study, physician self-reported practice change was corroborated with Evaluator observations of a subsample of study physicians in their offices. This is an innovative use of the simulated patient and enhances confidence in the results of the study [17, 18].

Our study has several potential limitations. Primary care physicians in the Washington, D.C., Metropolitan Statistical Area are not representative of all primary care physicians, and, therefore, the findings may have limited generalizability. Given a final participation rate of 47%, the study sample may not be representative of primary care physicians in the Washington, D.C., Metropolitan Statistical Area. An interaction effect between this bias and the study interventions would threaten generalizability of study findings. Nevertheless, we did not find differential effects of the interventions across study subgroups, and the demographic comparisons of study participants and nonparticipants showed no differences. Because this was a study to show effectiveness rather than efficacy, physicians who did not participate fully in the interventions were not dropped from the study and were not reassigned to other study groups.

The study group visited by an Instructor received the study intervention closer to the time of reassessment than did the other study group, which only received the educational materials, because the Instructor visited the physicians after the educational materials were mailed. Therefore, the study group visited by the Instructor may have been more "likely" to report changes in attitudes and practices in follow-up telephone interviews. It is unlikely that this could account for such consistent results across practice items in the self-report survey and across findings obtained by both physician self-report and unannounced Evaluator observation done an average of 2 months later. This consistency suggests that differences detected between the groups after the intervention are real and that changes associated with the Instructor persist at least for several months. It was not possible, however, to specifically study the extent to which the effect on practice decreased over time.

The Evaluators were predominantly white and all were female. Had they been a different race and sex, different results may have been observed. A separate report indicates that a single Evaluator is useful for evaluating the practice of groups of physicians but is unreliable for evaluating the performance of a single physician [20].

Although results from this study indicated that the Instructor intervention was helpful in increasing physician questioning regarding specific sexual practices and increasing physician counseling about condom use, many important aspects of prevention counseling for sexually transmitted diseases and HIV infection were not affected by the intervention. Specific details about how to use condoms and advice about safer sexual options in addition to intercourse were infrequently discussed by any study physicians when practice was observed. Both reported and observed risk questioning and counseling advice were incomplete, even among physicians trained by Instructors, which indicates a need for continued development of educational programs to further improve physicians' skills for risk assessment and counseling regarding sexually transmitted diseases and HIV infection.

The use of simulated patients and personalized feedback to train physicians is often used in medical education [14, 25, 26] but not often in continuing medical education [19, 27]. Because most primary care physicians are office-based and educational materials alone are ineffective in changing risk reduction counseling practices, programs that provide skills training, such as role-play with a simulated patient, are desirable. Further refinement of simulated patient training to reduce cost and facilitate access to training and development of other skills-based training approaches are needed to meet national goals for preventing sexually transmitted diseases and HIV infection by primary care providers.