RSS Feeds
Systematic Review: The Relationship between Clinical Experience and Quality of Health Care
- Niteesh K. Choudhry, MD;
- Robert H. Fletcher, MD, MSc; and
- Stephen B. Soumerai, ScD
- From Harvard Medical School and Harvard Pilgrim Health Care, Brigham and Women's Hospital, and the Harvard PhD Program in Health Policy, Boston, Massachusetts.
Abstract
Background: Physicians with more experience are generally believed to have accumulated knowledge and skills during years in practice and therefore to deliver high-quality care. However, evidence suggests that there is an inverse relationship between the number of years that a physician has been in practice and the quality of care that the physician provides.
Purpose: To systematically review studies relating medical knowledge and health care quality to years in practice and physician age.
Data Sources: English-language articles in MEDLINE from 1966 to June 2004 and reference lists of retrieved articles.
Study Selection: Studies that provided empirical results about knowledge or a quality-of-care outcome and included years since graduation or physician age as explanatory variables.
Data Extraction: We categorized studies on the basis of the nature of the association between years in practice or age and performance.
Data Synthesis: Overall, 32 of the 62 (52%) evaluations reported decreasing performance with increasing years in practice for all outcomes assessed; 13 (21%) reported decreasing performance with increasing experience for some outcomes but no association for others; 2 (3%) reported that performance initially increased with increasing experience, peaked, and then decreased (concave relationship); 13 (21%) reported no association; 1 (2%) reported increasing performance with increasing years in practice for some outcomes but no association for others; and 1 (2%) reported increasing performance with increasing years in practice for all outcomes. Results did not change substantially when the analysis was restricted to studies that used the most objective outcome measures.
Limitations: Because of the lack of reliable search terms for physician experience, reports that provided relevant data may have been missed.
Conclusions: Physicians who have been in practice longer may be at risk for providing lower-quality care. Therefore, this subgroup of physicians may need quality improvement interventions.
Improving Patient Care is a special section within Annals supported in part by the U.S. Department of Health and Human Services (HHS) Agency for Healthcare Research and Quality (AHRQ). The opinions expressed in this article are those of the authors and do not represent the position or endorsement of AHRQ or HHS.
Editors' Notes
Context
-
While “practice makes perfect” in some situations, physicians' knowledge and performance may decline with the passage of time.
Contribution
-
Of 62 published studies that measured physician knowledge or quality of care and described time since medical school graduation or age, more than half suggested that physician performance declined over time for all outcomes measured. Only 1 study showed improved performance for all outcomes measured.
Implications
-
This review should provoke careful study of the relationship of physician experience and the quality of care. It also raises concerns about the adequacy of continuing professional education in medicine.
–The Editors
Quality assurance and performance evaluation have become central issues in medicine. Care is suboptimal in many different medical conditions and clinical settings (1-6). Although delivering high-quality care is important to all clinicians, this issue may be particularly relevant to certain subgroups, such as physicians with less specialized training and those who see a smaller volume of patients (7-10).
Physicians who have been in practice for more years may also be less likely to deliver high-quality care (11, 12). Medical advances occur frequently, and the explicit knowledge that physicians possess may easily become out of date. Therefore, although it is generally assumed that the tacit knowledge and skills accumulated by physicians during years of practice lead to superior clinical abilities (13), it is also plausible that physicians with more experience may paradoxically be less likely to provide technically appropriate care.
Few existing studies have had the specific goal of evaluating the effects of experience on the quality of medical care (11). However, length of time in clinical practice has been included as part of a set of physician characteristics that might explain variations in quality or that may be confounders of the association between quality and other factors (13-18).
The purpose of this paper is to assess the robustness of the relationship between clinical experience and quality of care by systematically reviewing empirical studies. Although we define experience as the number of years a physician has been in practice, physician age and time in practice are highly correlated (11, 19, 20); therefore, for the purposes of this paper, we consider these variables to be interchangeable.
Methods
We searched MEDLINE (Ovid Technologies, 1966 to June 2004; English language) for terms describing physician experience (keywords: physician age, clinician age, physician experience, clinician experience), physician demographic characteristics (keywords: physician characteristics, clinician characteristics), practice variation (subject heading: physician's practice patterns), and performance in various domains (subject headings: clinical competence, health knowledge, attitudes and practice, outcomes assessment [health care]; keywords: knowledge, guideline adherence, appropriateness, outcomes). We retrieved potentially relevant articles and reviewed their reference lists to identify studies that our search strategy may have missed (Figure 1). We also searched our personal archives to identify additional studies. We included studies if they 1) were original reports providing empirical results; 2) measured knowledge, guideline adherence, mortality, or some other quality-of-care process or outcome; and 3) included years since graduation from medical school, years since certification, or physician age as a potential explanatory variable. We excluded studies if they described practice variation that is not known to affect quality of care (for example, assessed test-ordering behavior in clinical situations where optimal practice is unknown) or evaluated the performance of fewer than 20 physicians. For studies that examined several different end points, we included only those outcomes that are linked to knowledge or quality of care.
We used a standardized data extraction form to obtain data on study design and relevant results. We categorized studies into 4 groups on the basis of whether they evaluated knowledge (for example, knowledge of indications for blood transfusion), adherence to standards of care for diagnosis, screening, or prevention (for example, adherence to preventive care guidelines), adherence to standards of care for therapy (for example, appropriate prescribing), or health outcomes (for example, mortality). We classified the results of each study into 6 groups on the basis of the nature of the association between length of time in practice or age and performance: consistently negative, partially negative, no effect, mixed effect, partially positive, and consistently positive. “Consistently negative” studies were those for which all reported outcomes demonstrated a statistically significant decrease in performance with increasing years in practice or age. “Partially negative” studies showed decreasing performance with increasing experience for some outcomes and no association for others. We used similar definitions for “consistently positive” and “partially positive” studies. “Concave” studies found performance to initially improve with years in practice or age, then peak, and subsequently decrease.
We did not use formal meta-analytic techniques because the included studies used many different effect measures and some did not report parameter estimates.
Since studies based on self-reported practice may suffer from social desirability bias (21), we explored the effect of study quality on results by subcategorizing studies according to whether they measured outcomes with self-reports (that is, using surveys and interviews) or observed practice (that is, using chart audits or administrative data review). We also compared studies according to whether they performed multivariable modeling to adjust for patient and physician covariates. We used the Fisher exact test to compare the observed frequencies. We conducted all analyses with SAS, version 8.2 (SAS Institute, Inc., Cary, North Carolina).
Role of the Funding Source
The Harvard Pilgrim Health Care Foundation supported this study. It had no role in the design, conduct, or reporting of the study or in the decision to submit the manuscript for publication.
Results
Fifty-nine articles that reported data on 62 groups of relevant outcomes formed the basis of our analysis. Overall, 32 of the 62 evaluations (52%) demonstrated a negative association between increasing experience and performance (that is, performance decreased as experience increased) for all outcomes assessed; 13 (21%) reported a negative association for some outcomes but no association for other outcomes; 2 (3%) reported a concave relationship (that is, performance initially increased as experience increased, then peaked, and subsequently decreased); 13 (21%) reported no association; 1 (2%) reported a positive association (that is, performance increased as experience increased) for some outcomes but no association for other outcomes; and 1 (2%) reported a positive association for all outcomes assessed (Figure 2).
Knowledge
Twelve studies assessed the knowledge of practicing physicians, and all studies reported a negative association between knowledge and experience (Table 1). Studies by Ayanian and colleagues (7) and Salem-Schatz and colleagues (22) had large sample sizes, high response rates, and good sampling methods; used rigorous criteria to evaluate knowledge; and performed multivariate analysis.
Ayanian and colleagues (7) surveyed physicians to assess their beliefs about the survival benefit of therapies for acute myocardial infarction; the appropriate use of these therapies has been well-established in randomized, controlled trials. Specialists were more knowledgeable than generalists; however, after adjustment for this and other variables, physicians younger than 40 years of age were more likely to correctly believe in the value of therapies that improve survival (for example, thrombolytic agents, aspirin, and β-blockers). They were also significantly less likely to believe in the value of therapies that have been disproved (for example, prophylactic lidocaine) (P < 0.05).
Salem-Schatz and colleagues (22) interviewed surgeons and anesthesiologists to assess their knowledge of the risks associated with and indications for the transfusion of blood products. They found a highly significant negative association between knowledge and the number of years the physicians had been in practice (P < 0.001).
Adherence to Standards of Practice for Diagnosis, Screening, and Prevention
Twenty-four studies have assessed the appropriateness of physician use of diagnostic and screening tests, as well as preventive health care (Table 2). Overall, 15 (63%) of these studies demonstrated that physicians in practice for more years were less likely to adhere to standards of practice in this domain.
In the largest of these studies, Czaja and colleagues (33) surveyed participants to assess their adherence to cancer screening guidelines endorsed by the American Cancer Society and the National Cancer Institute. Physicians who had graduated more than 20 years before the survey were consistently less likely to adhere to recommended practices (odds ratio, 0.62 to 0.72; P < 0.05).
Using more objective measures of guideline adherence, Aubin and colleagues (17) assessed the practice of 21 physicians and found that after adjustment for patient and physician covariates, younger physicians were more likely to appropriately screen for hypertension (odds ratio, 1.11 [95% CI, 1.06 to 1.15).
Several other studies provide contrary results. Streja and Rabkin (47) assessed the use of recommended preventive care measures and found that after adjustment for other physician covariates (such as specialty, practice style, and number of diabetic patients in their practice), older physicians were more likely than younger physicians to test for proteinuria (odds ratio, 2.62 [CI, 1.61 to 4.37]) and to refer their patients for screening ophthalmology assessments (odds ratio, 1.48 [CI, 1.01 to 2.18]). However, older physicians were no more likely to order a high-density lipoprotein cholesterol level test. Their analysis did not adjust for any patient variables, such as the presence of macrovascular and renal disease. Rhee (12) evaluated the performance of 454 physicians treating patients in 15 different medical and surgical diagnostic categories and found a concave relationship between years in practice and adherence to standards of practice. Physicians in practice for 6 to 15 years provided the most appropriate care, whereas physicians with more or fewer years of experience provided less appropriate care.
Adherence to Standards of Appropriate Therapy
Table 3 presents the 19 studies that have assessed the influence of physician age and years in practice on adherence to standards of therapy. Of these studies, 14 (74%) found a partially or consistently negative association between physician age and adherence to standards of appropriate use of therapy.
A large and well-designed study by Beaulieu and colleagues (64) examined the prescribing behavior of physicians caring for patients with stable angina. After multivariate adjustment in a hierarchical model, older physicians were significantly less likely to prescribe aspirin (odds ratio for physicians in practice for > 20 years compared with those in practice < 10 years, 0.58). Age did not affect use of β-blockers or lipid-lowering agents.
Outcomes
Seven studies present data on the relationship between number of years in practice and actual health outcomes (Table 4). The strongest of these was conducted by Norcini and colleagues (14), who analyzed mortality for 39 007 hospitalized patients with acute myocardial infarction managed by 4546 cardiologists, internists, and family practitioners. After controlling for a patient's probability of death, hospital location and practice environment, physician specialty, board certification, and the volume of patients seen, these researchers observed a 0.5% (SE, 0.27%) increase in mortality for every year since the treating physician had graduated from medical school.
Hartz and colleagues (11) specifically assessed the association between experience and mortality rates for surgeons performing cardiac artery bypass grafting. After adjustment for both patient and physician variables, they found that physicians who have been in practice longer had higher operative mortality rates (P < 0.001). In contrast, Burns and Wholey's (69) large study of patients hospitalized for various conditions found no difference in mortality rates for physicians of different ages, but physicians in practice for more years had longer lengths of stay even after adjustment for patients' comorbid conditions and other physician factors.
Study Quality
To determine the influence of methodologic quality on study results, we stratified the 43 reports pertaining to adherence to standards of practice on the basis of whether outcomes were assessed by using self-reported data or more objective measures (that is, use of chart audits or administrative databases). Overall, 30 (70%) of these studies demonstrated a consistently or partially negative association between length of time in practice or physician age and adherence to standards of care. While the proportion of studies that found a consistently or partially negative association was slightly larger for self-reported studies than for those studies that used objective measures (71% vs. 62%), these differences were not statistically significant (P > 0.2).
Stratifying studies on the basis of whether they performed a multivariable analysis yielded similar results: 71% of the studies that adjusted for patient covariates found a consistently or partially negative association compared with 74% of studies that did not adjust for these factors, and 68% of the studies that adjusted for physician covariates found a consistently or partially negative association compared with 67% of studies that did not.
Discussion
Although based on heterogeneous studies, our systematic review of empirical studies evaluating the relationship between clinical experience and performance suggests that physicians who have been in practice for more years and older physicians possess less factual knowledge, are less likely to adhere to appropriate standards of care, and may also have poorer patient outcomes. These effects seem to persist in those studies that adjusted for other known predictors of quality, such as patient comorbidity and physician volume or specialization. The results are somewhat paradoxical since it is generally assumed that clinical experience enhances knowledge and skill and, therefore, leads to better patient care.
Our findings have many possible explanations. Perhaps most plausible is that physicians' “toolkits” are created during training and may not be updated regularly (70). Older physicians seem less likely to adopt newly proven therapies (71, 72) and may be less receptive to new standards of care (73). In addition, practice innovations that involve theoretical shifts, such as the use of less aggressive surgical therapy for early-stage breast cancer or protocols for reducing length of stay, may be harder to incorporate into the practice of physicians who have trained long ago than innovations that add a procedure or technique consistent with a physician's preexisting knowledge (74).
Our findings may also reflect the substantial environmental changes that have occurred in medicine over the past several decades; evidence-based medicine has been widely adopted, and quality assurance techniques, such as disease management and performance evaluation, are frequently used. More experienced physicians may have less familiarity with these strategies and may be less accepting of them. Given this, our results may represent a cohort effect; that is, when the current generation of more recently trained physicians has been in practice for a longer time, there may be smaller differences between their practice and those of their younger colleagues than our data would suggest.
Our study has several limitations. First, although we attempted to systematically review the literature on the association between number of years in practice or physician age and performance, our search strategy may have missed reports. This reflects the limited attention to this issue and the lack of consistent search terms to identify clinical experience. In addition, studies that were specifically designed to assess the relationship between experience and performance but found no association may have been less likely to be submitted or accepted for publication, and published studies that included number of years in practice or age among other physician characteristics may not have presented non–statistically significant results for these particular variables. Therefore, while we have no reason to suspect that we were more likely to identify studies showing decreasing performance with age, our findings are still potentially subject to an under-reporting bias.
Second, few reports included in this review were designed to specifically evaluate length of time in practice as their primary characteristic of interest. Consequently, our results may have been due to chance arising from multiple testing. However, we believe this is unlikely given the relative consistency of the results in several different domains, their “dose–response” relationship, and their overall plausibility. Moreover, restricting our analysis to the 32 studies that considered a broader set of physician characteristics, including number of years in practice or age as the focus of their investigation (that is, excluding those studies that considered physician age or number of years in practice only as confounders), does not change our results: 21 of the 32 (66%) studies reported a consistently or partially negative association between physician age and performance, whereas only 1 study demonstrated a partially positive association.
Third, disagreements may exist between clinical practice guidelines (33), and, thus, establishing appropriate norms may be difficult. As a result, assessing performance on the basis of guideline adherence may not reliably assess health care quality. Despite this, some studies included in our review used norms that had been adopted by several professional associations and that consequently reflect widely accepted standards of practice. Even for these studies, we observed age effects.
Finally, length of time in practice may be associated with other dimensions of quality that are not captured by the outcome measures that we evaluated. While we identified studies that assessed various conditions and aspects of performance, the relationship between age and performance may be different for other diseases and outcomes. For example, older physicians may be more effective at delivering the humanistic, rather than the technical, aspects of medical care. If this were true, one would expect that the patients of older physicians would report higher satisfaction, which has been demonstrated in some studies (75, 76) but not others (77, 78). Alternatively, physicians who have been in practice for a longer time may have better clinical judgment and may thus provide better care in complex cases or may be better diagnosticians. These outcomes have not been rigorously assessed.
Despite these limitations, our results are troubling. Although it is difficult to draw firm conclusions about the performance of older physicians in managing specific conditions or clinical scenarios, our results do suggest that older physicians may need quality improvement interventions that are generally applicable to all physicians. In addition, the requirements that are imposed on physicians to keep up to date and to demonstrate continuing competence should be further considered. Widely adopted continuing medical education techniques, such as the distribution of printed materials and lectures, are largely ineffective even in experimental conditions (79). Our results reinforce this. Moreover, many experienced physicians are exempt from the recertification requirements to which their more recently trained colleagues must adhere. For example, the American Board of Internal Medicine only requires physicians who received initial Board certification in or after 1990 to appear for periodic recertification examinations.
Our results also have implications for further research. The link between experience and performance should be further evaluated with studies that are designed a priori to specifically measure this association. These studies should use objective and widely accepted measures of performance; should be disease- or process-specific; and should be replicated for physicians of different specialties, demographic characteristics (such as sex), and different environment practices. The effect of age for physicians who routinely collaborate with other physicians, who frequently engage in evidence-based discussions, or whose practices are influenced by disease management, performance feedback, and computerized reminder systems may be different from that for physicians who practice in relative isolation or in more traditional settings.
An optimal study would follow a particular cohort of physicians over time. However, this is not practical and may be confounded by other secular trends in health care provision. Alternative designs would be similar to those of the highest quality included in our review and would adequately control for patient comorbidity, other physician factors, and the clustering of patients within physicians. These studies should also model the nature of the relationship between experience and performance since performance may improve during the initial phases of independent practice, plateau for some period of time, and then decrease. Finally, the ability of behavior change strategies to reduce the disparities in quality created by physician age should be evaluated in well-controlled clinical trials.
In summary, our results suggest that physicians with more experience may paradoxically be at risk for providing lower-quality care. The extent, magnitude, and nature of these results must be clarified, and added attention should be given to this subgroup of physicians who may need quality improvement interventions.
Improving Patient Care is a special section within Annals supported in part by the U.S. Department of Health and Human Services (HHS) Agency for Healthcare Research and Quality (AHRQ). The opinions expressed in this article are those of the authors and do not represent the position or endorsement of AHRQ or HHS.
Article and Author Information
-
Grant Support: By the Harvard Pilgrim Health Care Foundation. Dr. Choudhry is also supported by the Harvard Medical School Fellowship in Pharmaceutical Policy Research, a Frank Knox Scholarship from Harvard University, and a Canadian Institutes of Health Research Postdoctoral Fellowship. Dr. Soumerai is an investigator in the HMO Research Network Center for Education and Research in Therapeutics, funded by the U.S. Agency for Healthcare Research and Quality.
-
Potential Financial Conflicts of Interest: None disclosed.
-
Requests for Single Reprints: Niteesh K. Choudhry, MD, Department of Ambulatory Care and Prevention, Harvard Medical School, 133 Brookline Avenue, 6th Floor, Boston, MA 02215; e-mail, choudhry{at}fas.harvard.edu.
-
Current Author Addresses: Drs. Choudhry, Fletcher, and Soumerai: Department of Ambulatory Care and Prevention, Harvard Medical School, 133 Brookline Avenue, 6th Floor, Boston, MA 02215.
References
- 1.↵
- 2.↵
- 3.↵
- 4.↵
- 5.↵
- 6.↵
- 7.↵
- 8.↵
- 9.↵
- 10.↵
- 11.↵
- 12.↵
- 13.↵
- 14.↵
- 15.↵
- 16.↵
- 17.↵
- 18.↵
- 19.↵
- 20.↵
- 21.↵
- 22.↵
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.
- 29.
- 30.
- 31.
- 32.
- 33.↵
- 34.
- 35.
- 36.
- 37.
- 38.
- 39.
- 40.
- 41.
- 42.
- 43.
- 44.
- 45.
- 46.
- 47.↵
- 48.
- 49.
- 50.
- 51.
- 52.
- 53.
- 54.
- 55.
- 56.
- 57.
- 58.
- 59.
- 60.
- 61.
- 62.
- 63.
- 64.↵
- 65.
- 66.
- 67.
- 68.
- 69.↵
- 70.↵
- 71.↵
- 72.↵
- 73.↵
- 74.↵
- 75.↵
- 76.↵
- 77.↵
- 78.↵
- 79.↵
Related Clinical Content
