Many housestaff training programs have begun to adopt work shifts [3, 4] despite the lack of data showing beneficial effects of such schedule changes. In fact, Laine and colleagues [5] found in a retrospective cohort study that in-hospital complications and diagnostic test delays were more frequent after New York State enacted work-hour limitation regulations. One possible explanation for these findings is that such reforms increase the amount of time patients are covered by physicians who are not primarily responsible for their care (cross-coverage).

We studied the relation between housestaff coverage and potentially preventable adverse events in hospitalized patients using previously tested self-report methods [6].

Methods

Top Methods Results Discussion Author & Article Info References

Patients

All 3146 patients admitted to the medical service of Brigham and Women's Hospital, a 720-bed tertiary care urban teaching hospital, from 13 November 1990 to 14 March 1991 were eligible for the study. During this period, the medical service was staffed by 8 general medical teams, 2 cardiology teams, and 3 intensive care unit teams. Medical and cardiology teams consisted of 1 junior or senior resident, 2 interns, 1 or 2 medical students, and 1 attending physician. Each intensive care unit team consisted of 2 or 3 interns, 1 or 2 residents, and 1 attending physician.

Housestaff Call Rotations

Each day, 4 members of the general medical teams were on call for admissions for the next 24 hours. After approximately 6 p.m., 2 of the 8 medical residents and 4 of the 16 medical interns on the general medical service remained in the hospital, representing 4 teams. After midnight, 2 additional junior or senior residents acted as "night floats" on the general medical service. The night floats cared for patients of interns from teams not on call and for all patients and new admissions after 2 a.m. All newly admitted patients cared for by the night floats were transferred to the team that was on call the following day. The cardiology and intensive care unit services had no night floats, and those teams were covered all night by the on-call interns. Interns and residents were on call every fourth night on the general medical and cardiology services and every third night in the intensive care units. Because there were only two cardiology team residents, a rotating resident on call for the cardiac intensive care unit supervised the cardiology interns who were on call 2 of the 4 call nights.

Identifying Case-Patients and Controls

During the study period, the junior or senior resident physicians on each clinical team received daily electronic mail computer messages reminding them to report adverse events. An adverse event was defined as an injury caused by medical therapy that prolonged hospital stay or produced disability at discharge. This disability was not related to the underlying disease process and was not an intended consequence of medical therapy. The adverse event was the outcome measure used and tested in the Medical Practice Study in New York State [7], a large study of medical injury. The investigator trained residents in defining the outcome measure and reporting methods every 2 weeks when resident rotations changed.

Reporting adverse events was voluntary. Residents could report events by sending electronic computer mail, placing an anonymous card in a study box with the patient's name and location, or reporting the event in person to the housestaff working on the study. Preventability was not a criterion for reporting events.

After the name and location of the patient was transmitted to the study team, a resident working on the study completed a standardized adverse event data form and an Acute Physiology and Chronic Health Evaluation II (APACHE II) [8] data form for the potential case-patient and two controls. Data were collected prospectively before preventability was determined. The two controls were the patients in the beds on either side of the case-patient at the time of the event. We selected the controls to reflect the population at risk at the time of the event. Because a hospitalized patient could be in the risk group for more than one case-patient, our sampling design and analysis allowed some persons to be selected as controls for more than one case-patient or for a selected control to later be identified as a case-patient [9].

Only patients cared for on the medical service (that is, the general medical, intensive care unit, or cardiology service) were eligible to serve as controls. If no such patients were hospitalized on the same floor as the case-patient, the corresponding beds on the floor below the case-patient were used to identify controls, except for patients in the intensive care unit. These patients served as controls for each other but never served as controls for other services, and vice-versa. Patients were assigned to resident teams in a rotating manner and were distributed to hospital floors according to the availability of beds. Private patients were not delegated to separate wards or services. Coverage and admission routines were the same for all patients.

Assessment of Preventability

Because we sought to identify risk factors for potentially preventable adverse events, event summaries were reviewed by a panel of three physicians to assess preventability. This panel also ensured that the study criteria were met and classified adverse events as being related to medications, procedures, or other therapies. Summaries did not contain coverage data or insurance data. The reviewers first independently assessed the events and then met to develop consensus. The preventability of each event was rated on a 6-point scale. A score of 1 indicated little or no evidence for preventability, and a score of 6 indicated nearly certain evidence for preventability. Events that scored 4 or higher on this scale were judged potentially preventable. A "majority rules" criterion was used—at least two of three physicians on a panel had to agree on a preventability score ranging from 1 to 6, with the third panel member scoring the event within 1 point. Disagreements were resolved by discussion and further chart review.

As an example of a preventability rating, a patient with a new pleural effusion had thoracentesis by standard technique and suffered a pneumothorax that prolonged hospital stay. No evidence from the housestaff report or chart suggested any abnormality in procedure or technique. Preventability rating was 1 on a 6-point scale, and the event was classified as procedure-related. In another instance, a young woman with the acquired immunodeficiency syndrome was admitted for fever. The chest radiograph done at admission was read by the housestaff as normal, but later review indicated a pneumonia. Because of the error, antibiotic therapy was initiated after progressive pneumonia resulted in hypotension and admission to the intensive care unit. The misreading of the chest radiograph was classified as preventable by the panel, with a rating of 5. The panel believed that a system to ensure rapid review of radiographs on admission by a radiologist and transmission of the results could prevent such an occurrence. The event was classified as one caused by "other therapy."

In 68% of events, consensus of preventability was achieved by initial independent assessments. For the remainder, consensus was easily achieved by discussion and further review. Reproducibility of panel judgements of preventability was tested on a random sample of 45 adverse events, including 17 events initially classified as potentially preventable. These events were evaluated twice by the three reviewers at a 7-month interval. Analysis of agreement between the two sets of ratings by the panel yielded a statistic of 0.60, which is similar to the inter-rater reliability statistics reported in other studies of appropriateness and quality of care [10-15]. Further examples of adverse events, preventability ratings, and possible system changes to prevent some events are given in Appendix 1.

Data Collection

We collected demographic, socioeconomic, severity of illness, comorbidity, and procedure utilization data for case-patients and controls by reviewing the patients' charts and the hospital fiscal database. Sociodemographic information included age, sex, race, and type of insurance. The clinical variables recorded for each case-patient and two controls included resuscitation status, comorbid medical conditions (including the conditions listed in the Charlson comorbidity index [16]), and procedures and major therapies (for example, cardiac catheterization) that were done before the event. To measure severity of illness and to calculate a severity of illness score, we used APACHE II [8] (which is based on laboratory data, physiologic data, chronic health data, age, and the Glasgow Coma Scale [17]). We calculated the APACHE II score for both case-patients and controls from data collected immediately before the event.

We classified housestaff coverage data in two separate ways. First, the physician responsible at the time of event was identified from chart notes or time of day as 1) the primary intern, 2) another intern from the same team, 3) an intern from a different team, or 4) the night-float resident. Coverage during the adverse event was assigned by determining who was responsible for the patient when the incident that led to the event occurred. For example, if a patient with a known penicillin allergy was ordered to receive ampicillin, the event was attributed to the physician who wrote the order, regardless of when the allergic reaction was detected. Coverage at the time of the medication order would be determined for both the case-patient and matched controls in adjacent beds at the time the order was written.

We hypothesized a priori that patients who were covered by doctors not primarily responsible for their care were at higher risk for preventable adverse events, and we defined "cross-coverage" as management by an intern from a different team or by the night-float resident. We assumed that because the entire team (two interns and the resident) saw patients in rounds together, this exposure afforded some level of continuity of care to the patients covered by the same team. We calculated that patients on the medical service were cross-covered 30% of the time. Of this 30%, approximately 11% represented cross-coverage by the night-float resident, and 19% represented coverage by an intern from another team who was also on call for admissions. Because of the structure of the housestaff training program, no patients in the intensive care unit were ever covered by physicians from another team; therefore, no cross-coverage took place in the intensive care units.

The second management classification was by the day in the primary intern's call schedule, which was categorized as admitting day, postcall day, or "swing" day. Interns on an every-fourth-night rotation were thus on call (8 a.m. on the call day to 8 a.m. the following day) 25% of the time and were postcall (8 a.m. of the morning after call to 8 a.m. the following day) 25% of the time. An intern who was neither on call nor postcall was defined as being on a swing day. For control patients, we recorded the coverage status at the time of the case-patient's adverse event.

Statistical Analysis

We evaluated univariate correlates of adverse events using Mantel-Haenszel tests; each stratum corresponded to a matched set consisting of one case-patient and two controls. This test is the normal approximation for the more complicated exact test for matched data [18]. We did multivariate analyses using conditional logistic regression to identify independent correlates of adverse events. A conditional logistic regression model was adopted to control efficiently for any confounding factor attributed to the matching factors (location and time of day) [19]. In such analyses, odds ratios and confidence intervals are calculated by exponentiation of the regression coefficient.

Results

Top Methods Results Discussion Author & Article Info References

Resident Reporting

During the 4-month study period, 46 junior and senior residents rotated through the medical service and received daily computer reminders regarding the study. Of these 46 residents, 42 (91%) responded at least once, either reporting a potential event or indicating that no adverse events had occurred on their service. Thirty-nine (85%) physicians identified at least one event during the study period.

During the study, 3146 patients were admitted, and 124 adverse events were identified by the housestaff among 113 admissions (4%). Of these 124 adverse events, 50% were procedure-related, 31% were medication-related, and the remaining 19% were related to other therapies. The panel of reviewers gave preventability scores of 4 or more to 54 events (44%), which were therefore classified as potentially preventable.

Analysis of Potentially Preventable Adverse Events

Univariate Analysis

The 54 patients with potentially preventable adverse events were similar to their controls in age, sex, race, and the percentage residing in a nursing home (Table 1). Compared with their controls, the case-patients were more likely to have Medicare insurance and less likely to have commercial insurance or to belong to a health maintenance organization.

Several clinical factors had univariate associations with potentially preventable adverse events. Patients with potentially preventable adverse events were more likely (P < 0.05) than their controls to have hypertension (50% and 34%, respectively), a previous history of gastrointestinal bleeding (20% and 8%), chronic liver disease (9% and 1%), and a "do-not-resuscitate" order (20% and 8%). The mean APACHE II scores before the adverse event were 14.7 ±8.3 for case-patients and 8.8 ±4.6 for controls (P < 0.001), indicating that the patients who had adverse events were more severely ill than their controls. Case-patients were more likely than controls to have had a paracentesis (7% compared with 0%; P < 0.05) or colonoscopy (6% compared with 1%; P < 0.05) done before the event.

In the analysis of coverage status, potentially preventable adverse events of case-patients were not significantly associated with postcall periods when compared with those of matched controls (33% compared with 24%; P = 0.20) (Table 2). However, patients who had potentially preventable adverse events were more than twice as likely to be covered by an intern from another team or the night-float resident at the time of the event as were the matched control patients in adjacent beds (26% compared with 12%; P < 0.05) (Table 2). There was a nonsignificant trend toward a "protective" effect Table 2 of coverage by either the patient's admitting intern (odds ratio, 0.6; 95% CI, 0.3 to 1.3) or coverage by interns from the same team (odds ratio, 0.6; CI, 0.2 to 1.7).

To address the question of whether cross-coverage was a marker for the tendency of emergent procedures to occur at night, we reviewed all cross-covered patients to calculate the numbers of emergent procedures done during that period. In the cross-coverage periods, 1 of the 14 case-patients had an emergent procedure done for refractory chest pain. This finding is consistent with the lower diagnostic and therapeutic activity expected during cross-coverage periods.

To address the question of whether a bias in reporting could have produced the perceived association between cross-coverage and adverse events, we analyzed data from a chart review [6] of all 3146 patients admitted during the study period. The chart review of the cohort was done in a parallel arm of the study by physicians from another teaching hospital. The medical record analysts and physician reviewers in this part of the study were blinded both to the fact that any particular event was reported by the housestaff and to the coverage status of any particular patient at the time of an event because coverage information is obtained through the call schedule, not the chart. This review led to the identification of an additional 14 potentially preventable adverse events that were not reported by the housestaff. Of these unreported potentially preventable adverse events, 7 (50%) occurred during periods of cross-coverage by an intern from another team or the night-float resident, a proportion that was greater than the 26% of reported potentially preventable adverse events that occurred during cross-coverage periods. Therefore, it appears unlikely that under-reporting of potentially preventable adverse events accounted for the association between these events and cross-coverage.

Multivariate Analysis

The following univariate factors (significant at P < 0.10) were included in a conditional logistic regression model: cross-coverage by a physician from another team, APACHE II score, resuscitation status, type of insurance, history of gastrointestinal bleeding, hypertension, nursing home residence, end-stage renal disease, and cancer. In this model Table 3, only three risk factors—cross-coverage by an intern from a different team or by the night float, APACHE II score, and a history of gastrointestinal bleeding—were significant independent correlates of potentially preventable adverse events. Cross-coverage had an odds ratio of 6.1 (CI, 1.4 to 26.7), the APACHE II score had an odds ratio of 1.2 per point increase (CI, 1.1 to 1.4), and history of gastrointestinal bleeding had an odds ratio of 4.7 (CI, 1.2 to 19.0). The area under the receiver-operating characteristic curve constructed from this model was 0.78, suggesting good discrimination of case-patients from controls.

Analysis of Unpreventable Adverse Events

In univariate analyses, the 70 patients with unpreventable adverse events were similar to their controls in mean age, sex, race, type of insurance, and percentage residing in a nursing home. Case-patients differed from their controls in several clinical characteristics (P < 0.05), including the presence of hypertension (57% and 41%, respectively), history of gastrointestinal bleeding (14% and 4%), hypercholesterolemia (33% and 19%), and performance of angiography (39% and 17%) or angioplasty (19% and 3%). Mean APACHE II scores were 11.2 ±6.0 for case-patients and 9.8 ±5.8 for controls (P = 0.07).

We found no association between unpreventable adverse events and any of the housestaff coverage variables. In a global conditional logistic regression model, only one risk factor was a significant independent correlate of unpreventable adverse events: higher APACHE II score (adjusted odds ratio, 1.1 per point increase; CI, 1.0 to 1.2).

Discussion

Top Methods Results Discussion Author & Article Info References

Using previously validated methods, we did a case–control analysis to show the relation between housestaff coverage and adverse events while controlling for patient factors that might affect this relation. We found that patients' risks for potentially preventable adverse events were increased when the patients were cross-covered by physicians from another team, particularly when the cross-covering physician was an intern. This association was not present in the analysis of events judged unpreventable by reviewers, further supporting a causal link between discontinuity of care and potentially preventable events. These findings suggest that even a fatigued intern with detailed knowledge about a patient may be able to render more appropriate care than a well-rested one who is both less familiar with the patient and also busy admitting new patients to the hospital.

The data on housestaff coverage reported here are consistent with and extend findings from previous studies that have evaluated the effect of cross-coverage on patient outcome. One report [5] showed that complication rates and delays in diagnosis increased after a change in the structure of a housestaff program in which total working hours for interns were reduced but cross-coverage periods for patients were increased. Other reports have shown that transfer of responsibility for a patient's care is associated with increased laboratory testing and other resource use in both inpatients and outpatients [20-22]. However, our study differs from previous ones in its focus on preventable events, the prospective nature of the data collection, the extent and nature of clinical data and housestaff coverage data collected for case-patients and controls, and the use of concurrent control patients. Furthermore, our matched design avoids the potential bias that could be introduced by the lower intensity of clinical activity during cross-coverage periods.

Our findings suggest that changes in the call schedule, including limitations on physician hours, must be made with care. Such changes should have the same rigorous evaluation as other medical interventions. Limiting hours and increasing shift work increases cross-coverage. Cross-coverage can be complicated by discontinuity of care plans, incomplete transmission of information about patients, and subsequent errors in judgment by covering physicians who are unfamiliar with the details of the patient's case.

The width of the confidence interval reflects the limited power of this study to fully address the association between coverage by postcall interns and adverse events (odds ratio, 1.7; CI, 0.8 to 3.7). It is possible that resident fatigue may not have been a significant correlate of preventable adverse events because of preexisting changes in this housestaff program such as the night-float system. This system effectively gives the residents needed rest while apparently providing high-quality care (only two potentially preventable events were reported under night-float coverage.)

Two caveats related to cross-coverage attend our findings. First, we cannot exclude the possibility that residents may have been more likely to report events if potentially preventable adverse events occurred while the patient was under the care of a cross-covering physician. This "reporting bias" could lead to an apparent association between cross-coverage and adverse events. However, chart review of all 3146 admissions during the study period showed only 14 unreported preventable adverse events, and the proportion of these events that occurred during cross-coverage periods was actually higher than the proportion of the reported events. (Only 46% of all adverse events reported by the housestaff were also detected by chart review [6].) Therefore, we found the same result by using a method in which reporting bias was not likely. This suggests that bias did not cause the association between preventable adverse events and cross-coverage.

Second, teaching hospitals are extraordinarily complicated health care delivery systems, and teasing out the different factors that affect patient outcome is difficult and challenging. We cannot exclude the possibility that other variables, such as supervision or work load, may also be important in explaining our findings. However, the fact that housestaff coverage was not associated with unpreventable adverse events is further evidence of the importance of cross-coverage in this relation.

In addition to the cross-coverage finding, we found that increasing severity of illness was an independent correlate of both potentially preventable and unpreventable adverse events. This finding is consistent with results from the Medical Practice Study [23, 24], in which the complexity of disease or treatment was a major determinant of adverse events. Patients who are severely ill may have physiologic impairments that render them more vulnerable to adverse events. Patients who are more severely ill are also more likely to have more interventions and, with each intervention, increase their risk for adverse events.

We also identified an association between a history of gastrointestinal bleeding and adverse events. Patients with gastrointestinal bleeding are clinically more likely to have paracentesis and colonoscopy. Because of the few events, we could not control for these procedures in the multivariate model; therefore, gastrointestinal bleeding may well have served as their proxy in the analysis. Future quality improvement work will require review of these procedures and consideration of appropriate service changes, should they be indicated.

A possible limitation of our study design is related to the methods for measuring severity of illness and preventability of adverse events. Neither the Charlson comorbidity index nor the APACHE II scale were developed for predicting the risk for adverse events. After we considered other published, validated indices of comorbidity and severity of illness that might be appropriate for patients hospitalized at several different levels of care, we chose these instruments for their ability to permit residents to collect reproducible clinical data on many patients.

Finally, the generalizability of findings from this single teaching hospital remains uncertain. We used methods in which physicians were asked to identify adverse events in patients under their own care. The high participation rate (91%) was achieved because this investigation was a housestaff research project, and the interns and residents were assured that the details of individual events would not be shown to personnel not involved in the study. Because of these factors, these data collection methods may not be generalizable to other settings. Further, although the overall adverse event rate in this study was similar to that reported in the New York State Medical Practice Study, the types of adverse events reported here and their risk factors may be institution-specific. For example, the rate of complications after angiography in this cohort was 0.4%—considerably lower than the rate reported in other studies [25].

Our data indicate that the occurrence of potentially preventable adverse events is associated with cross-coverage by physicians who are less familiar with patients than their usual interns. However, the finding that cross-coverage is associated with an increased risk for potentially preventable adverse events should not be misinterpreted as a reason for reversing the trend toward reducing the time that residents spend on duty. Housestaff work-hour reforms are, in part, a response to several irreversible changes in American medicine, including decreasing lengths of hospital stay, increasing medical technologic sophistication, and increasing average severity of illness. Reforms represent in part a laudable attempt to make the training process more humane and to acknowledge the psychological stresses imposed by these recent changes. Indeed, housestaff surveyed in New York after work-hour limitations were enacted believed that they provided better care after the reforms [26], and program directors surveyed regarding night floats reported favorable changes in housestaff attitude and fatigue and increased ability to recruit new housestaff [27].

In our institution, while work-hour reforms proceed, we have instituted a series of interventions in conjunction with the housestaff that is designed to improve continuity of care. In our study and others, the interventions are based on a philosophy of preventing iatrogenic problems that requires a system-wide approach to analysis, not just a focus on identifying culpable persons [28-30].

Appendix: Examples of Adverse Events and Preventability Ratings

Case-patient 1: In preparation for coronary angiography, therapy with warfarin prescribed for atrial arrhythmias was discontinued, and heparin therapy was initiated. The patient developed prolonged chest pain after angiography that required intravenous nitroglycerine and admission to the cardiac care unit. After stabilization, the patient was transferred to a step-down unit, where heparin therapy was discontinued without reinitiation of warfarin therapy. The patient had a stroke, and the failure to restart anticoagulation therapy was considered the adverse event. The preventability rating was 5 on a 6-point scale because the panel felt that instituting a reminder system for clinicians regarding the indications for medications or improved continuity might have prevented the event. The event occurred during a cross-coverage period.

Case-patient 2: A patient with a new pleural effusion had thoracentesis by standard technique and suffered a pneumothorax that prolonged hospital stay. The preventability rating was 1 on a 6-point scale. The event occurred during the primary intern coverage period.

Case-patient 3: A 73-year-old woman with metastatic breast cancer and mild baseline congestive heart failure was admitted for elective transfusion. She received 3 units of red blood cells during 8 hours and developed congestive heart failure and electrocardiographic changes that required that her care be transferred to the step-down unit. The preventability rating was 5 because the panel felt that better monitoring of volume status during transfusion might have prevented the complication. This event occurred during cross-coverage.

Case-patient 4: A 62-year-old man with exacerbation of chronic obstructive pulmonary disease was given intravenous fluids and one dose of benzodiazepine in the emergency room before admission. A covering house officer was later called by the nurses for desaturation and ordered over the telephone an increase in the nasal cannula oxygen to 6 L/min. The patient became somnolent and had to be transferred to the intensive care unit for a PCO₂ of 101.0. The preventability score was 6 because the panel believed that the house officer might not have ordered the oxygen if a bedside evaluation was done, particularly in the setting of recent benzodiazepine use. The event occurred during cross-coverage.

Case-patient 5: A 64-year-old woman with acute myelogenous leukemia experienced daunorubicin cardiotoxicity complicated by congestive heart failure and volume overload. She received appropriate doses of chemotherapy. The panel graded the event as a 2 (unpreventable). The medication was ordered during the primary intern's coverage.