Proponents of resident research see it as a way to improve resident education, promote quality patient care, and provide essential skills for lifelong learning [2-5]. Others have suggested that resident research enhances analytic reading skills and critical thinking and that it can prepare graduates for various research roles in academia and the community [6-11].

Many residents value research training, and the absence of this training has been criticized by graduates of respected medicine training programs [12, 13]. In one university program, no other learning activity was rated as more important than the required research project, and 86% of the graduates and 66% of the senior residents agreed that all physicians should have research experience [14].

Despite the widespread appeal of resident research, there are many perceived barriers to it, including lack of mentors, lack of training opportunities, and lack of infrastructure [2, 15-19]. Other reported barriers include lack of resident interest, lack of curricular time, lack of background instruction, lack of financial support, and the pressures of clinical duties [20-22].

Most of the literature on resident research [2, 8, 10, 16, 17, 23-30] comes from disciplines without a strong research base, such as family medicine, psychiatry, and rehabilitation medicine. In these disciplines, resident research has important links to the viability of the disciplines themselves and to the promotion of academic careers, and it is a justification for reimbursement for services. Similar arguments have been echoed by leaders in subspecialty and general internal medicine [31, 32], and it has been proposed that internal medicine programs become more flexible to allow interested residents to obtain research experience [33]. To this end, the American Board of Internal Medicine established the Clinical Investigator Pathway, which permits board eligibility after 2 years of clinical training and 2 years of research training (34, pp. 8-10). Some programs, most notably that at Brigham and Women's Hospital and the Subspecialty Training and Research program at the University of California, Los Angeles, have developed parallel clinical and research tracks to launch young physicians on research careers [35, 36]. Research training initiatives have been proposed for junior faculty and general internal medicine fellowships [26, 37, 38], and yet the question of research experience for most residents has been largely ignored.

In the past decade, two studies pertinent to internal medicine have attempted to quantify research activity during residency. A 1991 survey of all graduate medical education programs [39] reported that more than 75% of these programs had research rotations and that 66% required these rotations. Almost half of the reporting programs required a research project, but subspecialty programs were more likely than programs accepting postgraduate year (PGY)-1 residents to require a project. A survey of internal medicine program directors in 1983 [40] found that 53% of programs offered a research elective, but no information was given about how many residents took advantage of this, what was offered, or what was accomplished.

Resident research is considered to be an important pedagogical skill vital to the growth and development of subspecialty and general internal medicine. Yet, basic questions remain unanswered, such as what constitutes resident research; what the requisite knowledge, skills, and attitudes are; and what resources are needed. In an attempt to answer these questions and to determine current readiness to meet the mandated research requirements, we describe 1) the current level of resident research activity; 2) the research environment and available resources; 3) educational outcomes and skills considered important; and 4) important barriers to resident research.

Methods

Top Methods Results Discussion Author & Article Info References

A 33-question survey was mailed to all ACGME-accredited internal medicine training programs in September 1993. The surveys were addressed to the program directors listed in the 1993 ACGME Directory of Graduate Medical Education Programs. Subsequent mailings were sent to nonresponders in November 1993 and February 1994. The questions were organized into four sections: research activities of categorical internal medicine residents; opinions about resident research activities; research activities of faculty; and residency demographic information. The following definitions were provided in the survey:

1. Hypothesis-driven research mandates an a priori establishment of a hypothesis, collection of data, and analysis of data with inferential or descriptive statistics.

2. Descriptive studies are observations not driven by a specific hypothesis and may consist of a single case report, a case series, or a description of a population.

3. Literature reviews do not involve the collection of original data or observations. Literature reviews may be analytical reviews that provide a comprehensive, critical assessment of the available published data on a medical subject, and may be subject to meta-analytical techniques. Nonanalytical reviews meet few or none of these criteria, but simply report on findings published in past and current literature without a critical, predesigned framework of appraisal or statistical analysis.

In section one of the questionnaire, 21 questions addressed aspects of resident research activities, including mandatory and minimal research expectations for residents; the presence, nature, and efforts of a research director; the presence, components, resources, and format of an organized research program; the presence and nature of protected time for resident research; the current level of resident research activity; and the educational and skill outcomes desired from resident research.

Three questions in section two elicited opinions about resident research, asking directors to state the three most important barriers to that research; the three most important reasons that residents engage in research; their level of agreement with the idea of mandatory research activities; and the ability of their programs to implement such activities. Four questions in section three addressed the expectations for and activity of faculty members with regard to research and the availability and suitability of faculty members as resident research mentors. Five questions in section four collected demographic information on the training program, including total faculty number, faculty number by type (full-time or volunteer), and university affiliation.

The survey instrument was piloted in six different residency programs, and the results of the pilot were included in the final results. The surveys were completed anonymously.

Survey responses were divided into two mutually exclusive groups: those from university-based programs and those from non-university-based programs. University-based programs were defined as programs administered by a department of medicine at a medical school or as community programs integrated with university programs. Non-university-based programs were all other programs, including those at Veterans hospitals, community hospitals [university-affiliated or independent], military hospitals, health maintenance organizations, and large, multispecialty clinics. We chose to divide programs this way because 1) we believed that university-based programs were more likely to have resources pertinent to research and 2) we wanted our data to be comparable with that in previously published studies.

Because we surveyed all ACGME-accredited internal medicine training programs, we made a finite population correction before calculating Student t-test results. Percentages and chi-square analyses were used to compare categorical data, and means ±SDs and Student t-tests were used to compare continuous data. Ranked categorical variables were weighted. First ranking was awarded 3 points, second ranking was awarded 2 points, and third ranking was awarded 1 point. Weighted mean scores were calculated by dividing the total score for an item by the number of responders. For each item, the differences in weighted means for university-based and non-university-based programs were compared using the Student t-test. For all analyses, the level was set at 0.05. Results are given as means ±SD.

Results

Top Methods Results Discussion Author & Article Info References

Of the 415 surveys mailed, 271 were completed and returned, yielding a response rate of 65%. A telephone survey of 10% of randomly selected non-responders was completed, and significant differences between nonresponders and responders were noted. The data were similar for university-based and non-university-based training programs, but those statistically significant differences that did exist are noted. Otherwise, the data reflect the overall trends of all training programs combined. Between 253 and 271 responders answered any given item, and the data are presented as percentages of responders answering a question rather than as percentages of all responders. For most questions, responders were allowed to choose more than one answer; therefore, the cumulative response rate may exceed 100%.

The distribution of training programs in the survey was as follows: 55% were at university-affiliated community hospitals; 35% were at university hospitals; 21% were at Veterans hospitals; 18% were at community hospitals integrated with a university; 6% were at community hospitals and were not affiliated or integrated with a university; 6% were at military hospitals; 4% were at health maintenance organizations; and 10% were "other." Many responders selected more than one category; thus, the total exceeds 100%. On the basis of these self-descriptions, 141 programs (52%) were chosen to compose the university-based group, and 130 (48%) were chosen to compose the non-university-based group. As expected, university-based training programs tended to be larger than non-university-based training programs: 22 ± 12 compared with 13 ± 9 PGY-1 residents (P < 0.001); 20 ± 11 compared with 12 ± 7 PGY-2 residents (P < 0.001); 19 ± 10 compared with 11 ± 7 PGY-3 residents (P < 0.001); and 3 ± 5 compared with 2 ± 3 PGY-4 residents (P = 0.009). Nonresponders had fewer PGY-4 residents (2.3 ± 4 compared with 1.1 ± 0.7; P = 0.001).

Are medicine training programs prepared to meet the new scholarly requirements? The data show that 97% of training programs have established expectations consistent with the new accreditation guidelines. Fifty-one percent agreed that a single case report would fulfill their minimum research expectations; other activities that fulfilled minimum expectations included doing an analytical (41%) or nonanalytical (40%) literature review; describing a series of cases (29%); doing hypothesis-driven research (26%); and describing a population (22%).

Program directors were asked to report resident research activities for the 1993-1994 academic year. These activities included hypothesis-driven research (18% ± 22%), nonanalytical literature reviews (18% ± 30%), analytical literature reviews (15% ± 27%), describing a series of cases (9% ± 12%), and describing a population (6% ± 14%). University-based programs were less likely than non-university-based programs to report a single case report as a resident activity (24% ± 26% compared with 32% ± 29%; P = 0.003). Nineteen percent of programs reported no research activity by their residents in the 1993-1994 academic year.

Program directors were asked to estimate the percentage of their residents involved in research by year of training. Less research activity was reported among PGY-1, PGY-2, and PGY-3 residents in university-based settings, with overall rates of 13% ± 28% compared with 19% ± 30% for PGY-1 (P = 0.01); 36% ± 32% compared with 46% ± 35% for PGY-2 (P < 0.001); and 55% ± 36% compared with 62% ± 35% for PGY-3 (P = 0.025) residents. University-based and non-university-based programs did not differ in the percentage of PGY-4 residents involved in research (56% ± 42%). Nonresponders reported more PGY-1 residents engaged in research activities (39% ± 38% compared with 17% ± 28%; P = 0.038). Program directors projected that 62% ± 36% of their residents would complete an acceptable research project by the end of the 1994 academic year.

Program directors were asked to estimate the percentage of their residents presenting or publishing during the 1993-1994 academic year. Fewer university-based residents presented their research locally (32% ± 31% compared with 44% ± 37%; P < 0.001) or at state or national meetings (18% ± 23% compared with 24% ± 27%; P < 0.02). No differences were reported in the percentage of residents who would publish their research in the 1993-1994 academic year. Nonresponders reported fewer peer-reviewed publications (4% ± 4% compared with 11% ± 14%; P < 0.001) but more local oral and poster presentations (58% ± 38% compared with 39% ± 39%; P = 0.05).

Program directors were also asked about research leadership, academic and technical support, and mentoring. Leadership, in the form of a research director, was present in 38% of programs. Among those programs that had research directors, 61% of the directors were medical doctors; 18% held a medical degree and another advanced degree (PhD, MPH, or MS); 17% held an advanced degree and no medical degree; and 4% had other titles or academic degrees. Forty-two percent of research directors spent 0% to 10% of their efforts in teaching or in coordinating or directing resident research activities; 38% devoted 11% to 20% of their efforts to these activities; 15% devoted 21% to 40% of their efforts to these activities; and 4% devoted 41% to 60% of their efforts to these activities.

Academic support was assessed by the presence of a comprehensive research curriculum or of specific curricular components generally acknowledged to be pertinent to learning research skills. A comprehensive curriculum was present in only 38% of programs, but most residencies taught, either as a mandatory or a voluntary experience, many of the topics or skills important to research. Overall, 93% of programs taught critical appraisal skills, 95% taught bibliographic retrieval skills, 85% taught computer skills (word processing, graphics, or statistical analysis), 78% taught research methods, 69% taught epidemiology, 59% taught survey design, 69% taught biostatistics, 74% taught medical informatics, and 71% taught scientific communication skills (writing, poster design, or oral presentation skills). Nonresponders reported less instruction in computer skills (57%; P = 0.029) and medical informatics (43%; P = 0.029). Only critical appraisal skills were mandatory in more than 50% of programs. University-based programs were more likely to teach critical appraisal skills (P = 0.001) and biostatistics (P = 0.03) but were less likely to make these experiences mandatory (Table 1). Journal clubs were the most common teaching format (73%) and were followed by lectures (56%) and longitudinal seminars (33%).

Various technical resources were available at no direct cost to residents in most training programs. Eighty-four percent of programs provided access to personal computers, 76% had statistical software available, 68% provided research design consultants, 53% provided writing consultants, and 68% provided visual and graphic design consultants. University-based programs were more likely to have statistical consultants (82% compared with 70%; P = 0.02). Nonresponders reported less visual and graphic consultant support (57%; P = 0.05). Only data collection support (research technicians, research nurses, and students) was unavailable or provided at cost in more than 50% of programs.

Research mentors were available in 96% of programs. The percentage of faculty serving as mentors was similar in university-based and non-university-based training programs, but university-based programs had a higher percentage of "academically prepared" mentors: Seventy-seven percent of university-based programs and only 40% of non-university-based programs reported that 41% to 100% of their faculty were academically prepared to serve as mentors. The definitions of "mentor" and "academically prepared" were not provided in the survey but were left to the discretion of the program directors.

Protected time for research was present more frequently in university-based programs (77% compared with 63%; P = 0.01). Of the programs that provided protected time, 98% provided an elective research rotation. University-based programs offered slightly more time (7.7 ± 6.0 compared with 6 ± 3.3 weeks; P = 0.01). Other forms of protected time included a few hours each week for the duration of the research project (11%) and time during portions of certain elective rotations (35% of university-based programs and 51% of non-university-based programs; P = 0.04).

When rating educational outcomes, 92% of program directors rated "becoming a more critical consumer of research" as a very important or somewhat important outcome. Between 57% and 86% of directors rated "learning research skills," "completing a research project," and "contributing new knowledge" as very important or somewhat important (Table 2). Nonresponders were less likely to rate "completing a research project" or "contributing new knowledge" as important or somewhat important outcomes (76% compared with 57% [P = 0.05] and 57% compared with 28% [P = 0.038], respectively).

Program directors were asked to consider the types of careers that their residents typically enter and then to rate the importance of selected research skills to their graduates. Ninety-five percent of directors rated critical appraisal of medical literature as very important or somewhat important. Between 53% and 84% of directors rated the ability to do nonanalytical or analytical literature review, to describe a population, to describe a series of cases, to describe a single case report, and to do hypothesis-driven research as very important or somewhat important (Table 3).

Program directors were asked to select the top 3 barriers to resident research from a list of 10 barriers frequently cited in the literature (Table 4). The top 2 barriers chosen were "lack of resident time" and "lack of resident interest." The third was "lack of money" for university-based programs and "lack of faculty" for non-university-based programs. Compared with non-university-based programs, university-based programs gave relatively higher-weighted scores to "lack of resident time" and lower-weighted scores to "lack of faculty time." Program directors were also asked to choose, from a list of seven reasons, the top three reasons why residents do research (Table 5). The highest mean weighted scores from both university-based and non-university-based programs were assigned to "improve fellowship applications." University-based program directors assigned the next highest weighted scores to "satisfy intellectual curiosity" and "research is mandatory"; non-university-based program directors made the same selections, but in reverse order.

Finally, the attitudes of program directors toward mandatory research requirements and the ability of their programs to meet these requirements were assessed. Fifty-four percent of directors strongly agreed or agreed with the statement "Resident research activity should be mandatory for internal medicine training programs," but 53% also strongly agreed or agreed with the statement "It will be difficult for our program to meet the RRC [Resident Research Committee] guidelines requiring research activity by all residents."

Discussion

Top Methods Results Discussion Author & Article Info References

This was a nonrandom survey of 271 (65%) internal medicine residency program directors. In an attempt to estimate how representative the responders were, a random sample of 10% of the nonresponders was surveyed by telephone. The differences between responders and nonresponders were relatively few and minor, but caution should be used when attempting to generalize our results to all program directors.

Most internal medicine training programs appear to be on course to meet the new accreditation requirements for resident scholarly activity, and university-based and non-university-based training programs differ little in this regard. Of some concern is the finding that 19% of programs reported no research activity by residents in the 1993-1994 academic year, the year before the implementation of the new accreditation guidelines.

Our study documents that most medicine training programs have in place many of the elements associated with scholarly productivity, including a mentored research experience, structured training in research methods, and a work environment that supports research [37, 41]. Studies of research productivity in residency programs have stressed the importance of choosing a mentor, faculty guidelines about mentoring responsibilities, academic credit, the development of clear goals and objectives, the presence of support personnel, and a classroom curriculum [22, 28, 42-45].

Our study documents that research mentors are present in almost all training programs, but the identity of these persons and the nature of their experiences are unclear. More "academically prepared mentors" were associated with university-based programs, but the significance of this finding is difficult to interpret, because program directors may have different definitions of "academically prepared." Having more specific information about research mentors and mentoring may prove helpful to programs wishing to improve their residents' scholarly productivity.

Structured training in research is considered important, but little has been published on what should be taught. Some direction has been provided by Neinstein and MacKenzie [46], who surveyed the first authors of papers published in 18 leading peer-reviewed journals. Two thirds of the responders to their survey recommended courses in basic biostatistics, research design, computer use, epidemiology, and health statistics. Similar topics have been suggested by educators associated with successful resident research programs [43] and graduates of internal medicine programs [14, 25]. Our findings are consistent with these recommendations: More than 70% of the training programs we surveyed taught the nine curricular topics that we asked about. Only two topics, critical assessment and biostatistics, were found more frequently in university-based than in non-university-based programs. Although our study can suggest what should be taught, it does not address whether resident research contributes to the quality of the residency program, to career selection, or to improvements in graduates' practice of medicine. These are important issues worthy of future research.

Although it was not specifically cited as an essential component of resident research, most training programs appear to have adequate technical support. More than two thirds of programs reported the presence of six of the eight technical resources that we asked about in our survey. With the exception of statistical consultants, university-based programs did not provide more technical resources than non-university-based programs.

The provision of protected time for resident research has increased from 53% 10 years ago [34] to 69% today; university-based programs offer the most protected time. The absence of protected time is frequently cited as a barrier to resident research [21, 22, 47, 48], but the presence of protected time has not been consistently associated with increased productivity [22, 27, 28, 42, 49]. Productivity is also related to the presence of structured training, mentoring, and a supportive environment in which research is valued. In the absence of these variables, protected time alone is unlikely to enhance productivity.

Two previous studies [5, 42] have suggested that resident research is increased when the training program is closely related to a medical school or university. Our results do not completely support this contention. Non-university-based programs exceeded university-based programs in the total number of residents involved in research activities and in the number of mandatory components of a research curriculum. University-based programs reported having more academically trained research mentors and more protected research time for their residents. No significant differences were found between the publication rates of residents in university-based and non-university-based programs, but residents in non-university-based programs were more likely to present the results of their research in local, state, and national settings. These results need to be interpreted cautiously, because they are based on responder recall rather than on direct measures of productivity.

There was a remarkable degree of unanimity among program directors when they rated the importance of educational outcomes and skills related to research. More than 70% of all program directors rated "becoming a critical consumer of research" and "critically reading research" as very important. Again, our results need to be interpreted cautiously, because they reflect the opinions of program directors rather than of residents, who are the ultimate beneficiaries of the educational program. Nevertheless, the program directors' opinions are supported by the results of a survey of internal medicine graduates [14], who reported that the required research project (and morning report) were the most effective means of teaching critical reading skills.

More than half of the responders doubted their ability to meet the research accreditation guidelines. What can be done? First, program directors need to recognize that no universal treatment exists for teaching research skills. Each program must decide which type of training is in the best interests of their graduates. Hitchcock and Buck [11] have proposed a spectrum of research roles for physicians. In their model, tenure-track physicians do research and serve as mentors for students and residents. Clinical faculty members provide an academically stimulating environment and serve as role models of sound practice behaviors. Practitioners participate by forming collaborative relationships, and they collect information on their patients that can be analyzed centrally. Being a consummate consumer of medical research overlaps all research roles.

When this model is applied to internal medicine training, the result is that residents interested in academic careers will need an introduction to research training during residency and the opportunity to practice their skills under the guidance of an experienced mentor. For the trainee interested in a tenure-track position, further research training should be obtained through subspecialty or general medicine fellowships or through faculty development fellowships, such as those supported by the Division of Medicine, Health Services Research Administration, or the Robert Wood Johnson Clinical Scholars Program. In general, we propose that residents heading for a medical practice career will benefit more from research education rather than training, provided that it is sufficient to allow them to critically evaluate the work of others.

The pessimism expressed by program directors is not entirely congruent with our data. The ingredients for research training or education are present in most residencies. Program directors have identified and teach concepts and skills that are likely to provide lifelong educational benefits for most of their graduates. Program directors do need to be concerned about the relevance of their research curricula and need to ensure that the educational prescription they apply is the one indicated for their graduates.

Dr. Anderson: Office of Medical Education, Research and Development, A217 East Fee Hall, Michigan State University, East Lansing, MI 48824.

Dr. Albrecht: Office of Medical Education, Research and Development, A212 East Fee Hall, Michigan State University, East Lansing, MI 48824.

Dr. Poland: General Internal Medicine and the Mayo Vaccine Research Group, 601 B Guggenheim, 200 First Street SW, Rochester, MN 55905.