Failure to Recognize Newly Identified Aortic Dilations in a Health Care System With an Advanced Electronic Medical Record

Methods

The institutional review boards of the Iowa City and Omaha Veterans Affairs Medical Centers approved this project.

Data Collection

We obtained radiology reports from consecutive patients who underwent CT of the abdomen and pelvis during 2003 in any of 2 midwestern Veterans Affairs medical centers. The reports contained 3 sections: the indications for the study (for example, abdominal pain), a detailed reading of the scan, and the radiologist's summary of important findings. As of 2003, all CT reports and clinical notes were stored in the Veterans Affairs EMR–Computerized Patient Information System. Although each of the medical centers generally followed established radiology guidelines for CT reporting, including when the radiologist should contact clinical teams directly regarding unanticipated study findings, neither medical center had firm protocols in place for what did or did not constitute an unanticipated finding, as is commonplace in clinical practice today.

We used the CT data to search for patients who had abdominal aortic abnormalities in the infrarenal or suprarenal regions. We identified these aortic dilations by looking for 3 general phrases that would broadly encompass the language that radiologists use when describing the aorta with the assistance of a practicing radiologist member of the investigative team; these terms were “aort,” “aneurysm,” and “AAA.” Two of the investigators then performed keyword searches of the CT reports by using these terms. We intentionally did not limit our search to more specific phrases, such as “aortic aneurysm,” because a variety of radiologic criteria are used to define aortic aneurysms; therefore, radiologists may differ greatly in deciding which aortic abnormalities qualify as aneurysms as opposed to more general terms, such as “ectasias” or “dilations” (12). Likewise, we intentionally did not apply rigid size criteria (for example, 2.8 cm or 3.0 cm) to aortic abnormalities, because radiologists commonly use discretion when applying size thresholds to individual patients and before deciding which aortic abnormalities are clinically concerning (13).

We reviewed the radiology report for each patient with a dilation and entered the following information into a Microsoft Access (Microsoft, Redmond, Washington) database: the patient's last name and last 4 numbers of the Social Security Number, date of and indication for the CT (classified as asymptomatic screening, symptoms or clinical concern, follow-up of known AAA, or incidental [any dilation discovered on CT not ordered specifically for signs or symptoms, screening, or follow-up of a known AAA]), maximum abdominal aorta size, the descriptive terms the radiologist used to describe the dilation (categorized as ectasia, aneurysm, aneurysmal, AAA, or other), and location in the report where the aortic dilation was mentioned (detailed reading or summary). We also collected information from the radiology report on whether the radiology team contacted the clinical team about the aortic abnormality that was identified.

After reviewing the CT reports, at least 1 study investigator reviewed the complete EMR of each patient with a new aortic dilation to search for documentation suggesting recognition of the aortic abnormality and to examine subsequent patient management. We reviewed all data sources in what is widely considered to be one of the most advanced and complete EMRs in the United States. In particular, for each patient, we reviewed all inpatient and outpatient clinical notes, all nursing notes, imaging and operative reports, and summary lists of each patient's medical problems. To improve the accuracy of our review, we also performed a keyword search of all clinical encounters maintained in the EMR by using the same terms that were used in the review of the CT reports.

The medical record review commenced with verification that the aortic dilation detected by CT was not identified previously—we excluded patients with previously identified dilation. The EMR of all patients determined to have new abnormalities underwent full medical record review to collect an array of information, including patient demographic characteristics (age, race, and sex), aneurysm risk factors (tobacco use, hypertension, and family history of aneurysms), and significant active comorbid conditions (for example, end-stage heart failure, dementia, metastatic cancer, or renal failure) that might make a patient's aortic dilation irrelevant in the opinion of their health care providers; such patients were excluded from further analysis. We also collected data on the frequency of each patient's contact with the Veterans Affairs health care system for 1 year after the initial CT. In particular, we recorded whether each patient had any follow-up contact with the provider who ordered the initial CT and the total number of clinic visits and telephone contacts that each patient had during follow-up.

For all eligible patients, we collected information from the EMR about how much time elapsed between the initial CT until evidence appeared that a clinician recognized the aortic dilation. We considered a dilation recognized if the abnormality was explicitly mentioned anywhere in the EMR or if there was evidence of a relevant consultation (for example, vascular surgery) or follow-up imaging (for example, abdominal ultrasonography or magnetic resonance imaging). We considered a dilation missed only if the EMR showed no documentation of recognition or evidence of monitoring or treatment of the dilation. We also collected information about the clinical setting in which the initial CT was ordered (outpatient or clinic visit, inpatient, or emergency department visit) and the type of provider who ordered the CT, including their educational training (physician, nurse, or nurse practitioner), specialty (primary care, surgery, or other), and whether the ordering provider was a resident or staff physician.

Statistical Analysis

We used univariate methods (mean, median, and SD) to describe the demographic characteristics, aneurysm risk factors, and amount of contact with the Veterans Affairs health care system that patients with new aortic abnormalities had. We used bivariate methods (t test, Fisher exact test, or analysis of variance) to examine whether the radiology reports differed between patients with smaller abnormalities (<5.5 cm) and patients with larger abnormalities that would typically warrant consideration of surgical repair (≥5.5 cm).

On the basis of discussion among the investigative team, which included internists, a radiologist, and a vascular surgeon, we made an a priori decision to define timely recognition of a new aortic dilation as an abnormality for which we could find evidence in the EMR of recognition by the clinical team within 3 months of the index CT. We used bivariate methods to explore the relationship between timely recognition and important clinical factors available in our data set (for example, aorta size, clinical setting in which the initial CT was ordered, ordering provider, and whether patients had follow-up contact with the provider who ordered the CT). We performed sensitivity analysis by using alternative definitions of timely follow-up (for example, recognition of the aneurysm within 6 months) to examine the robustness of our findings. We also performed a time-to-event analysis in which the outcome of interest was recognition of the aortic abnormality and in which patient deaths were accounted for through censoring.

To ensure the validity of our results, we performed 15% of abstractions in duplicate, including all aortic abnormalities for which the first reviewer found no evidence of clinician recognition in the EMR. All reviewers agreed (κ = 1.0) on the presence or absence of aortic dilations and documentation of recognition of dilations within 3 months of the index CT. Agreement was somewhat lower for other variables, including the setting in which the CT was ordered (κ = 0.89) and the presence or absence of tobacco use (κ = 0.71). All analyses were conducted by using Stata SE, version 10.0 (StataCorp, College Station, Texas).

Role of the Funding Source

This project was funded in part by grants from the National Institutes of Health; the Robert Wood Johnson Foundation; and the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service. The funding agencies played no role in the study design or the analysis or interpretation of results.

Results

We reviewed the CT scans of 4112 patients and identified 440 (11%) with abdominal aortic abnormalities (Figure 1). We excluded 323 patients in whom CT was ordered to follow up on aortic abnormalities that were recognized previously and 26 patients with severe comorbid illness. Our final cohort consisted of 91 patients (2.2% of all CT recipients) with new aortic dilations (Table 1). The mean size of the aortic dilations was 3.5 cm (SD, 1.1; range, 2.2 to 9.3 cm); 83 (91%) of the abnormalities were incidental findings on CT, 6 (7%) were identified on CT ordered because of clinical concern for an aneurysm, and 2 (2%) were identified through routine screening.

View larger version:

• In this window
• In a new window

Figure 1. Study flow diagram.

CT = computed tomography.

In reviewing the CT reports, we found that radiologists infrequently notified the clinical teams of the aortic abnormalities and that notification was no more common for larger abnormalities than for smaller abnormalities (Table 2). Radiologists used diverse terms to describe aortic abnormalities, but terminology did not systematically differ for larger and smaller abnormalities. Moreover, findings that were documented in the CT summary often did not appear in the detailed radiology reading. For example, 27% of the CT reports used the term “aneurysm” in the detailed reading, whereas 46% of the CT reports had the term “aneurysm” in the summary.

Of the 91 patients with new aortic dilations, 85 (93%) had an identifiable source of primary care in the Veterans Affairs health care system. These patients had an average of 3.9 clinic visits and 0.9 telephone contact with the Veterans Affairs health care system within 3 months of their CT and 12.8 visits and 3.5 telephone contacts within 1 year. Only 4 patients had neither a clinic visit nor telephone contact with the Veterans Affairs health care system within 3 months of their CT, and only 1 patient had neither a clinic visit nor telephone contact within 1 year. In our survival analysis (Figure 2), the median time to documentation of recognition was 237 days. The EMR contained no evidence that abnormalities in 16 patients (29% of surviving patients) were recognized by clinical teams during a mean follow-up of 3.2 years.

View larger version:

• In this window
• In a new window

Figure 2. Kaplan–Meier curve for time to recognition of aortic dilations.

We compared the characteristics of abnormalities that were recognized within 3 months of the index CT with those that were not recognized within 3 months (Table 3). Abnormalities recognized within 3 months were significantly larger than abnormalities that were not recognized within 3 months (P < 0.001); however, 9% of abnormalities that were not recognized within 3 months were 5.5 cm in diameter or larger. The site where the CT was ordered, type of provider ordering the CT, and specialty of the provider ordering the CT did not differ appreciably for results that were or were not recognized within 3 months.

Overall, 59% of patients had contact within 1 year (clinic visit, documented telephone call, or letter) with the provider who ordered the CT, whereas 41% of patients did not have documented contact with the ordering provider. The proportion of patients who had follow-up contact with the provider who originally ordered the CT (Table 3) was similar for patients whose abnormalities were and were not recognized within 3 months. Findings were similar when we defined timely follow-up by using alternative criteria (for example, within 6 months or 1 year).

Discussion

In a retrospective cohort study conducted in an integrated health care delivery system with a sophisticated EMR, we found that new aortic dilations were often recognized by clinicians only after substantial delay and that 18% of dilations were never documented during an average follow-up of more than 3 years. However, it is somewhat reassuring that we found no evidence of patient harm as a result of the delays that were identified. Our findings highlight the need for innovative solutions to ensure that abnormal findings are consistently recognized and documented by clinical care teams.

Our results should be considered in light of the definitions we used to determine what constitutes recognition of an aortic dilation. We defined an aortic dilation as recognized if either of 2 conditions were met: the dilation was explicitly mentioned in the EMR, even if no clinical care plan was explicitly described, or there was evidence of action (for example, referral of the patient to a vascular surgeon) in the absence of explicit mention of the aortic abnormality. An aneurysm was considered unrecognized only if neither condition was met.

In our study, aneurysms might have gone unrecognized for 2 reasons: They may have been missed by the clinical team, or they could have seemed to be unrecognized because of errors in documentation or omission of documentation (12, 14). The difference between missed and undocumented aneurysms is more than an issue of semantics. Missed test results have the potential to harm patients, whereas incomplete documentation practices have significant medical and legal implications (15, 16).

Several additional findings merit further discussion. We found that fewer than 40% of patients with new aortic dilations had evidence in their medical record that the abnormalities were recognized by clinicians within 3 months of discovery, and 18% of patients lacked documentation of follow-up during an average of 3.2 years. In addition, more than 40% of patients with new aortic dilations identified on CT had no follow-up contact with the provider who ordered the index CT, suggesting a potential mechanism for results being missed. At the same time, only 0.3% of all patients receiving CT (16 of 4112 patients) had an undocumented aortic abnormality, and no patients died or had aneurysm rupture.

Our findings should also be considered in light of existing guidelines for management of aortic aneurysms. Although the average size of missed abnormalities was 3.4 cm, 9% were 5.5 cm or larger. Current clinical guidelines generally recommend that patients with aneurysms 3 cm or larger undergo annual follow-up imaging and patients with aneurysms 5.5 cm or larger be considered for surgical repair (17–19).

Our finding that 60% of incidental aortic dilations lacked evidence of recognition in the EMR by the clinical service within 3 months and that 18% were never documented should be considered in the context of studies of test result management. Studies conducted in multiple settings found that approximately 1% of elevated potassium levels, 2% of abnormal thyroid-stimulating hormone levels, and 23% of abnormal bone density CT scans were not recognized by providers according to medical record review (2, 3, 11, 20). Alternatively, Roy and colleagues (4) found that clinicians overlooked approximately 60% of important abnormal test results that returned after patients were discharged from a large teaching hospital, whereas Choksi and colleagues (21) found that 2% of “significant unexpected” radiology findings in the Ann Arbor Veterans Affairs medical center would have been lost to follow-up if not for a safety-net system they had developed. The wide variation in the proportion of unrecognized test results being reported is probably related to several factors, including variation in the tests being considered, the definition of an abnormal result, and the quality and completeness of the medical records (and provider documentation) that were available for review.

It is essential to consider the setting in which we conducted our study and the degree to which our findings can be generalized to other health care systems. In recent years, quality in the Veterans Affairs health care system has been shown to meet or exceed quality in the private sector (22). The Veterans Affairs health care system has an advanced EMR that is used by all Veterans Affairs hospitals and clinics. The EMR captures all clinical encounters, test results, and procedures and allows providers to access information on patients seen in any Veterans Affairs hospital or clinic across the country. Although we would encourage researchers and patient safety officers to examine follow-up of test results in their own institutions, the lack of comprehensive EMRs capturing both radiology results and all clinical encounters outside of the Veterans Affairs health care system would make it difficult to replicate our analyses outside of a few select integrated health care delivery systems. Of note, the Veterans Affairs health care system has been on the leading edge of patient safety for many years and recently funded a patient safety center at the Houston Veterans Affairs medical center that is specifically charged with improving test result management. At the same time, the Veterans Affairs health care system faces challenges that may worsen problems with documentation and follow-up of abnormal test results compared with the private sector, including frequent involvement of medical trainees and the service of a patient population with higher comorbidity and lower socioeconomic status. That said, a growing body of literature suggests that follow-up of test results is similarly challenging outside of the Veterans Affairs health care system, where EMRs are less available and investment in patient safety is reduced.

Our study has limitations. First, it was conducted in a rural midwestern network of Veterans Affairs hospitals and clinics and should be extrapolated to other settings and patient populations with care. Nevertheless, our findings of undocumented test results are similar to those of other studies conducted in other settings. Thus, we would encourage investigators and administrators who believe our findings to be atypical to repeat our analyses in their own settings. Second, our study was conducted by using retrospective chart review, and we could not assess aspects of care that were not documented in the medical record; this may have caused us to underestimate the rate of radiology communication with the clinical service or our inability to detect aortic dilations that were recognized by clinical services but not documented in the medical record. However, we were careful to give clinicians credit for recognizing aortic abnormalities if they mentioned the dilation or if follow-up care was initiated (for example, further imaging) in the absence of discussion of the dilation in clinic notes.

In conclusion, we found that a substantial proportion of new aortic dilations were not accompanied by documentation of clinician recognition in the EMR. Our findings highlight the need for better strategies to ensure test result follow-up.