RSS Feeds
-
In Response
Submit responseIn response to Dr. Quartin and colleagues’ concerns regarding misclassification of cases in our paper, we offer the following comments and further analysis.The Project IMPACT database includes the date and time of ICU admission and discharge, but only the date for starting and stopping CCM. Thus Dr. Quartin et al are correct that the PI database does not contain precise information about the initiation of critical care management during the first calendar day. This is a limitation of the study which often occurs when using an administrative database. We have acknowledged this in the discussion section of our paper (Third limitation of the study).
We would argue however that this limitation does not represent a misclassification of patients. We classified patients precisely according to the definition of CCM used by PI:
"The starting and ending dates of critical care physician management recorded in the database coincided with the dates of ICU admission and discharge."
Consider for example, a patient who was admitted to the ICU at noon and was subsequently managed by a critical care trained physician from 6 pm on the initial day until 7 days later when he was discharged. We would classify this patient as belonging in the CCM group. It certainly would not be appropriate to classify him in the non-CCM group.
Next, consider the issue for the aggregate of patients classified as having CCM their entire stay. If we assume that patients are admitted around the clock, the average time of admission would be half way through, or 12 hours, into the 24 hour period of a calendar day; and if some of these patients are transferred to CCM physicians, this transfer takes place on average half way through the remaining 12 hours of the day. Thus, on average, those patients transferred to a critical care trained physician on the first calendar were likely to receive an average of 6 hours of care under the supervision of a doctor not trained in critical care.
The question then becomes, how to interpret our results given the definition we use. How much does it matter if we do not know precisely when in the first 24 hours in the ICU management by critical care physicians begins? We ought to and do acknowledge that sicker patients are likely to be transferred to critical care trained physicians. The modified SAPS II score and propensity score are designed to adjust for the increased likelihood that critical care trained physicians care for sicker patients.
In order to address the possibility that some of the sickest patients are more likely than others to get transferred to CCM some time after admission on the first calendar day, we ran an analysis eliminating those admissions that were in the top decile of expanded SAPS II which would represent the top 10% of the sickest patients. The overall results do not change (Table 1). In addition, we ran our primary analysis without 10% of the shortest ICU LOS's (Table 2) and again without 25% of the shortest stays (Table 3). Deleting the 10% of patients with the shortest ICU stay, is equivalent to dropping those patients whose ICU stay is less than 0.66 days. Deleting the 25% with the shortest ICU stay, is equivalent to dropping out those whose ICU stay is less than 0.98 days, which should therefore delete all patients cared for by another physician. The results without the 10% short stays are almost identical to the results in the original manuscript. For the 25% shortest stays, the results are similar; however, the p-values are slightly attenuated. We believe that the results of these additional analyses demonstrate that even without including patients who may have been cared for by non-CCM physicians for a brief period of time, the results of the study are unchanged.
Thus despite the limitation of the dataset which does not allow us to know which patients received delayed critical care management on the first calendar day, this additional analysis that excludes the sickest patients who might be most likely to be transferred or patients with a very short length of stay does not change our major finding. We believe this addresses the concern about bias due to the transfer of such patients.
We do agree with the point that the modified SAPS II score does not offer any information about trends in a patient’s condition during the 24 hour period of the first day. In eliminating the patients with the worst scores from this latest analysis, patients who are sickest at either the beginning or the end, or for that matter at any point of the day, are excluded. Since the score is based on the worst values over the 24 period, it seems to us that it suffices to capture information regarding who is most likely to be transferred to the care of a critical care trained physician at any point in the first calendar day.
We comment on one last point made by Quartin et al, who mention that ".. the exclusion from analysis of patients who required transfer to critical care management after the first ICU day effectively removes high risk patients from the non-intensivist cohort." We would suggest that by excluding these patients we reduce the likelihood of these sicker patients being assigned to either the CCM or non-CCM group. Hence this exclusion does not seem to bias in favor of the non-CCM group.
We appreciate the comments by Quartin et al and the opportunity for further the dialogue on the important issue.
-
In Response
Submit responseWe are concerned that the authors may have misconstrued the meaning of key data fields from the Project IMPACT database. Both their article and response letter indicate they used these fields to assign patients into the cohorts they compared. Far from being an arcane issue, such misclassification would prevent any meaningful interpretation of the data.
In our first Letter to the Editor, we pointed out that patients classified as “managed entire stay by critical care physicians” may have in fact received up to 24 hours of non-intensivist directed care in the ICU prior to critical care physician management, depending upon what time of day ICU admission took place. This is very important. When transfer to critical care physician management occurs some time after ICU admission in “open” units, it is often motivated by poor response to initial therapies administered in the ICU, a harbinger of increased mortality. Inadvertent inclusion of these patients in the “managed entire stay by critical care physicians” cohort would constitute an obvious bias favoring management by non-intensivists.
In their response posted electronically on July 28, 2008, the authors do not dispute that the inclusion of such patients would bias their analyses. Rather, they incorrectly claim “the category ‘managed entire stay by critical care physicians’ exclude (sic) any patients managed by non-critical care physicians in the first 24 hours.” Their manuscript and response letter are not explicit with regard to which data elements from Project IMPACT were used to accomplish this. However, as we explain below, the data fields alluded to by the authors do not permit such distinctions, nor are we aware of others that would.
We note (and the authors concur in their response) that Project IMPACT records the dates but not the times critical care physician management starts and stops. Apart from individual patient data, Project IMPACT also records certain policy information for each participating ICU. Project IMPACT classifies patients as having “Critical Care Management During the Entire ICU Stay” under two circumstances:
1. The ICU the patient was admitted to has a policy dictating that all patients receive critical care physician management throughout their ICU stays.
2. The starting and ending dates of critical care physician management recorded in the database coincided with the dates of ICU admission and discharge.
All patients that Levy, et al. classified as having been cared for in a “choice” unit and “managed entire stay by critical care physicians” must have met this second criterion, since patients meeting the first criterion were cared for in ICUs that should have been classified as “no-choice”. Note that the second criterion is satisfied if critical care physician management starts on the same calendar day as ICU admission, even if there was a period of non-intensivist guided ICU therapy preceding critical care physician involvement on the ICU admission day.
Levy, et al. report that there were 23,324 patients “managed entire stay by critical care physicians” in “choice” units, over half the intensivist managed population. It is impossible to determine how many of these patients were transferred to critical care physician management after ICU admission but within the first calendar day, rather than admitted directly to the ICU under critical care physician management. However, as preferential transfer of difficult ICU patients to intensivists is common practice in many medical centers, we suspect this was not rare. In fact, the authors themselves excluded from analysis 17% of the patients managed by intensivists in “choice” units because of transfers that occurred after the first ICU day, which Project IMPACT does capture.
Accuracy in cohort assignment is very important for the analysis the authors conducted. Absolute differences between predicted and observed mortality rates in all cohorts were well under 2%. Shifting a few “sick” patients from one group to another can easily determine the outcome of analysis. Simulations have shown that performance relative to standardized mortality ratios, as the authors reported, are very sensitive to the addition or removal of just a few patients (e.g. Kahn, et al., Chest 2007; 131:68-75). (Moreover, the exclusion from analysis of patients who required transfer to critical care management after the first ICU day effectively removes high risk patients from the non-intensivist cohort, reinforcing this bias.)
Unfortunately, neither of the parameters used to adjust for confounding factors, a modified SAPS II score and a propensity score, can correct for misclassification. Both are insensitive to trends following ICU admission, the very factors often motivating transfer of management responsibility. SAPS II, based on worst physiologic values over the first 24 hours of intensive care, can yield identical scores for patients who are deteriorating, improving with initial therapy, or remaining static. For example, SAPS II assigns the same score for blood pressure derangement (five points) to both a patient arriving to the ICU with a systolic blood pressure of 85 mm Hg that than rises to 105 mm Hg after one liter of fluid, and a patient whose blood pressure falls from 105 mm Hg at admission to 85 mm Hg despite administration of several liters of fluid and institution of a norepinephrine infusion. Despite identical scores, the latter patient, with volume refractory shock, is both more likely to die and to be referred to a critical care physician. The propensity score was based entirely on factors apparent before ICU admission, and thus cannot capture or account for trends following ICU admission that might stimulate transfer to critical care physician management.
This paper has been a source of great interest and controversy, both within and beyond the Critical Care Medicine community. It is essential that the methodology employed be both appropriate and clearly delineated. If the authors used additional techniques to avoid inaccurate cohort assignment of the type we have described above, they have not indicated so in either their manuscript or in their initial response to our comments. Could the authors either more fully explain their methodology, or, alternatively, acknowledge a fundamental difficulty of their data?
Conflict of Interest:
None declared
-
Assault on reason
Submit responseIt was with great enthusiasm that we turned our attention to the article in the recent Annals of Internal Medicine (Levy MM, Rapoport J, Lemeshow S, Chalfin DB, Phillips G, Danis M. Association between critical care physician management and patient mortality in the intensive care unit (Ann Intern Med. 2008 Jun 3;148(11):801-9.) regarding the association between critical care physician management and patient mortality in the ICU. We were immediately struck by the similarity to the methodology used in the Connors study (1) which condemned the use of the pulmonary artery catheter, and which is largely responsible for many practitioners reducing their use of this technique. If indeed we are to accept this “propensity score” as a legitimate tool for evaluating the worthiness of a therapeutic approach, then we must condemn with equal enthusiasm the use of critical care physicians in critical care medicine. This obviously flies in the face of reason, and requires that we rethink the validity of the use of propensity scores in medicine. (1) The effectiveness of right heart catheterization in the initial care of critically ill patients. SUPPORT Investigators.Connors AF Jr, Speroff T, Dawson NV, Thomas C, Harrell FE Jr, Wagner D, Desbiens N, Goldman L, Wu AW, Califf RM, Fulkerson WJ Jr, Vidaillet H, Broste S, Bellamy P, Lynn J, Knaus WA. JAMA. 1996 Sep 18;276(11):889-97. Conflict of Interest:None declared
-
The Authors Response
Submit responseWe agree with and appreciate several of the thoughtful comments offered by authors of these letters. Most of the comments by these authors have been mentioned in the discussion section in our manuscript. We would like to make some important points for clarification: 1) Recalibration was necessary for the severity model. For this reason, an expanded SAPS II was utilized for severity adjustment in our study. We believe that Figure 1, along with the reported value for the Hosmer-Lemeshow goodness of fit statistic, in our published manuscript demonstrates the usefulness of the expanded SAPS II for predicting actual mortality in the population used in our study. 2) Critical care-managed patients did not receive any care from non critical care physicians. The category “managed entire stay by critical care physicians” exclude any patients managed by non-critical care physicians in the first 24 hours. Because Project Impact records dates but not times of critical care physician management, and in order to eliminate potential confounding by this uncertainty, all patients managed partially by critical care physicians were excluded from analysis in the study. We agree that the ability to detect unrecognized confounding by severity is limited by the information contained in the Project Impact database. 3) As stated in our discussion, it is true that the Project Impact database does not allow for the separation of “full time intensivistt” management from management by board certified critical care physicians who are not necessary “on-site.” Therefore, the “being there” factor was not evaluated in our study, nor was the impact or role of management by trainees. 4) Although the published literature suggest that full-time, on-site, intensivists are more likely to write for Do-Not-Resuscitate orders for critically ill patients, there was no difference in DNR orders amongst the groups evaluated in our study. 5) Patients in coronary care units were excluded from analysis in our study. 6) The study by Diringer et al. included patients from institutions outside of the Project Impact database and so does not have relevance to the current study. 7) The Project Impact database does not describe the “culture” of the ICUs included in the database. It is possible that the ICUs in which there is no management by critical care physicians are not representative of the typical open ICU in the United States. We welcome all the comments included in these letters as part of a new, ongoing dialogue aimed at improving the care provided to critically ill patients. Conflict of Interest:None declared
-
Benjamin Franklin on ICU mortality
Submit responseThe study by Levy et al makes interesting reading(1). Hospital mortality may be higher for those managed by intensivists, but mortality in the community is still 100%. As Benjamin Franklin put it, “In this world nothing is certain but death and taxes”.
1. Levy, M. M., J. Rapoport, et al. (2008). "Association between Critical Care Physician Management and Patient Mortality in the Intensive Care Unit." Annals of Internal Medicine 148(11): 801-809.
Conflict of Interest:
None declared
-
Hospitalist effect?
Submit responseTo the editor:
I read with interest Levy et al's analysis of of the effects of critical care medicine in a recently-collected large retrospective database. (1) Much of the previous work in this area had been before the advent of the hospitalist movement and with before-and-after study designs. (2)
Is it possible that much of the previously observed difference in mortality seen with intensivist staffing might not be present in situations where critical care is provided by hospitalists who do so regularly?
An added on-the-job critical care certification for hospitalists would be one way to meet the upcoming shortfall of critical care physicians.
Thank you.
1. Levy MM, Rapoport J, Lemeshow S, et al. Association between critical care physician management and patient mortality in the intensive care unit. Ann Int Med. 2008;148:801-9. 2. Pronovost PJ, Angus DC, Dorman T, et al. Physician staffing aptterns and clinical outcomes in critically ill patients: a systematic review. JAMA. 2002;288:2151-62.
Conflict of Interest:
None declared
-
In Response
Submit responseWe were intrigued by the results of Levy and colleagues(1) demonstrating higher odds for hospital mortality in patients managed by critical care physicians. We recently employed the Project IMPACT database to update the Mortality Probability Model (MPM) . Although our sample of 124,885 patients at 98 hospitals between October 2001 and March 2004 is not identical, the overlap is substantial. We analyzed the variable: “Percentage of patient stay managed by critical care physician/team”. In our sample, 68179 patients (54.6%) were coded as 0% critical care management (CCM), 50694 (40.6%) were coded as 100% CCM, and 6012 (4.6%) had a value between 0 and 100%. Mortality was universally higher in the CCM group at every decile of risk. However, the majority of patients (64% of non-CCM and 54% of CCM) fell into the lowest decile of MPM-III mortality risk where risk-adjusted outcome was exactly as expected (Standardized Mortality Rate (SMR) 1.0; 95% confidence interval 0.95-1.05 for both non-CCM and CCM managed patients). There were also no differences in SMRs in patients when the MPM0-III risk prediction was >50%. SMR differences occurred at the 2nd and 3rd deciles of MPM- predicted risk, exactly where the calibration curve for this model deviates from “perfect” prediction(2). Although our analysis utilizes MPM, we suspect similar calibration considerations may apply to SAPS used by Levy et al, and a calibration curve of their observed vs. predicted mortality, along with measures of model fit would be informative.
We agree with the authors that unmeasured confounders are likely. The pattern of raw mortality rates across severity-adjusted deciles of risk suggests case-mix differences between CCM and non-CCM units. The risks of emergency surgery are incompletely captured by risk adjustment models(3), and our Project IMPACT sample shows the proportion of emergency surgical patients to be 58% higher in CCM units. We suspect that other differences (urban vs. rural, teaching vs. non-teaching, alcohol and tobacco use, socioeconomic status) also play roles; more time is needed to evaluate these concerns.
As payment for medical care becomes linked to performance metrics, it is essential for these metrics to be accurate. Levy and colleagues have opened an important research question with their seemingly counter- intuitive results. More work is needed to find and correct for potential confounders before concluding that existing severity-of-illness models adequately correct for patient population differences in retrospective studies where assignment to CCM and non-CCM units is not random.
References:
1. Levy MM, Rapoport J, Lemeshow S, Chalfin DB, Phillips G, Danis M: Association between Critical Care Physician Management and Patient Mortality in the Intensive Care Unit. Ann Intern Med 2008; 148: 801-809.
2. Higgins TL, Teres D, Copes WS, Nathanson BH, Stark M, Kramer AA: Assessing contemporary intensive care unit outcome: An updated Mortality Probability Admission Model (MPM0-III). Crit Care Med 2007; 35:827-835.
3. Higgins TL, Brennan M, Jodka P, Teres D, Nathanson B: Elderly patients without MPM-III risk factors fare surprisingly well. Crit Care Med 2007; 35:12S; A171.
Conflict of Interest:
Drs. Higgins and Nathanson have been consultants to Cerner Corporation which administers the Project IMPACT database and supported the development of MPM-III
-
Association between critical care physician management & patient mortality.
Submit responseThe paper by Levy et al looking at relations between critical care and patient mortality (1) has a fundamental methodological weakness. The method they followed was that they first collected the data, second, they noted as association between critical care physician management & increased patient mortality, thirdly, they conducted further detailed & exceedingly complicated analysis with statistical experts on the same data-set & they thereby “verified” this adverse association. Nobel prize winning physicist Richard Feynman has lucidly pointed out the fallacy of this practice. In his lecture “This Unscientific Age” (2) he states “I now turn to another kind of principle or idea, and that is that there is no sense in calculating the probabil¬ity or the chance that something happens after it hap¬pens. A lot of scientists don't even appreciate this. In fact, the first time I got into an argument over this was when I was a graduate student at Princeton, and there was a guy in the psychology department who was run¬ning rat races. I mean, he has a T- shaped thing, and the rats go, and they go to the right, and the left, and so on..............................This man had designed an experiment which would show something which I do not remember, if the rats always went to the right, let's say, .....................And then he noticed, most remarkably, that they alternated, first right, then left, then right, then left. And then he ran to me, and he said, "Calculate the probability for me that they should alternate, so that I can see if it is less than one in twenty." I said, "It proba¬bly is less than one in twenty, but it doesn't count." He said, "Why?" I said, "Because it doesn't make any sense to calculate after the event. You see, you found the pecu¬liarity, and so you selected the peculiar case."
Levy et al, based on observed data, generated a hypothesis; There is an association between critical care physician management & increased patient mortality. On expert review with sophisticated statistics they essentially regenerated the same hypothesis. Instead of testing this hypothesis in another prospectively collected data-set, they interpreted the re-stated hypothesis as being a conclusion.
Considering the potential implications of this association between critical care physician management & increased patient mortality, I wonder if the editorial board was hastily inappropriate in accepting for publication instead of asking for prospective verification on a new data- set.
References
1. Mitchell M. Levy, MD; John Rapoport, PhD; Stanley Lemeshow, PhD; Donald B. Chalfin, MD, MS; Gary Phillips, MAS; and Marion Danis, MD Association between Critical Care Physician Management and Patient Mortality in the Intensive Care Unit. Ann Intern Med. 2008;148:801-809. 2. Richard Feynman. This Unscientific Age (Danz Lectures, Delivered in April 1963) in The Meaning Of It All. 2007 ed Penguin, London pp 61-122
Conflict of Interest:
None declared
-
Patient Course and Referral to the Intensivist
Submit responseLevy et al. retrospectively analyzed the Project Impact intensive care unit (ICU) database. They report intensivist management was associated with increased mortality, despite adjustment for severity of illness and propensity for intensivist management[1]. As the authors acknowledge, their counterintuitive results contradict prior studies. We suspect inadequate adjustment for confounding factors underlies these findings.
Most ICU severity models, including SAPS II and MPM0-II, were derived from populations with higher mortality than Project Impact’s. Without recalibration, these models exaggerate risk for low acuity patients. Thus, MPM0-III, developed from the same Project Impact database, required a novel “zero factor” term connoting negative risk to accommodate the inclusion of so many patients without discernible mortality risk[2]. Intensivists were less likely to manage low risk patients so any overestimation of risk for “zero factor” patients would favor non- intensivist management.
A second, crucial problem arises from an inability to distinguish patients improving with initial care from those not. Most patients were in units with “optional” intensivist management where transfer of deteriorating patients from non-intensivists to intensivists is common practice. SAPS II utilizes the worst physiologic derangements during the first 24 hours of ICU care, allowing patients worsening after ICU admission to have identical scores to those who improve rapidly. The problem is substantive: “intensivist managed” patients may have received up to 24 hours of initial non-intensivist directed ICU therapy, as Project Impact recorded dates but not times of intensivist management. The propensity score, determined by pre-ICU factors, is similarly blind to clinical trends. Since deterioration is both associated with mortality and motivates transfer to intensivists, the absence of data permitting adjustment for early ICU courses may be insurmountable.
This would not be the first study of decision-making for ICU patients where overlooking trend data led to suspect conclusions. Connors, et al. reported excess mortality with pulmonary artery catheterization[3], a finding unsupported by subsequent randomized trials[4,5]. As with the current study, adjustments ignored post-admission clinical trends, which doubtless impacted catheterization decisions. Though Levy et al. argue only large, undetected confounders could negate their findings, the attribution of mortality to pulmonary artery catheters missed confounders of such magnitude, in a similar population, under similar circumstances.
The distribution and SAPS II calibration for “zero factor” patients is addressable. Unfortunately, Project Impact does not contain the critical data needed to understand the larger issue of why intensivist management was chosen when optional. Its absence renders the study’s conclusions doubtful.
1. Levy MM, Rapoport J, Lemeshow S, Chalfin DB, Phillips G, Danis M. Association between Critical Care Physician Management and Patient Mortality in the Intensive Care Unit. Ann Intern Med. 2008; 148(11):801-9.
2. Higgins TL, Teres D, Copes WS, Nathanson BH, Stark M, Kramer AA. Assessing contemporary intensive care unit outcome: an updated Mortality Probability Admission Model (MPM0-III). Crit Care Med 2007; 35:827-35.
3. Connors AF Jr, Speroff T, Dawson NV, Thomas C, Harrell FE Jr, Wagner D, et al. The effectiveness of right heart catheterization in the initial care of critically ill patients. JAMA. 1996; 276(11):889-97.
4. Harvey S, Harrison DA, Singer M, Ashcroft J, Jone CM, Elbourne D, et al. Assessment of the clinical effectiveness of pulmonary artery catheters in management of patients in intensive care (PAC-Man): a randomized controlled trial. Lancet 2005; 366:472-7.
5. National Heart, Lung, and Blood Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network. Pulmonary-artery versus central venous catheter to guide treatment of acute lung injury. N Engl J Med 2006; 354:2213-24.
Conflict of Interest:
None declared
-
Challenges Facing Critical Care
Submit responseThe June 2008 study in Annals by Levy and colleagues found an association between critical care management and increased mortality. This follows the recent publication of the Dartmouth Health Atlas 2008(1) and associated commentary in the New England Journal of Medicine.(2) The Dartmouth study reviewed the records of several million Medicare patients hospitalized during the last two years of life at nearly 3000 hospitals. Both the Levy and Dartmouth studies offer unfavorable appraisals of critical and end-of-life care as practiced today.
Citing wide disparities in end-of-life-care expenditures, the authors of the Dartmouth study argue such differences arise from a lack of unified scientific standards. For example, Medicare patients in their last two years of life at UCLA were cared for by an average of 16.9 physicians with an average cost of $93,842 dollars. This contrasts with similar patients at the Mayo Clinic who had 8.9 physicians with an average cost of $53,432. The authors of the Dartmouth study argue higher intensity end-of- life-care, much of which occurs in the ICU, has increased costs without clearly improving outcomes and that wide disparity in expenditures amongst institutions reflects a lack of evidenced based medicine rather than disease severity or patient preference. Similarly, the authors of the Levy study suggest that the increased mortality with critical care consultation reported in their study may stem from a lack of evidence based protocols and over-reliance on potentially risky procedures.
Like any research, these studies should not be accepted without careful and skeptical analysis. But regardless of their merits, the field of critical care is likely to remain a focus of policy makers and researchers when you consider that critical care expenditures are estimated at $55 billion annually.(3) Widespread application of evidence- based and cost-effective care is our best hope for preserving and enhancing the status of critical care medicine as a field, yet challenges clearly remain.(4-5)
1.The Dartmouth Health Atlas Executive Summary. Accessed at http://www.dartmouthatlas.org/atlases/2008_Atlas_Exec_Summ.pdf on June 15, 2008.
2.Goodman DC, Fischer ES. Physician Workforce Crisis? Wrong Diagnosis, Wrong Prescription. N Engl J Med. 2008;358:1658-1661. [PMID: 18420498]
3.Halpern NA, Pastores SM, Greenstein RJ. Critical care medicine in the United States 1985–2000: An analysis of bed numbers, use, and costs. Crit Care Med. 2007 Mar;35(3):692-9. [PMID: 17255850]
4.Dennison CR, Mendez-Tellez PA, Wang W, Pronovost PJ, Needham DM. Barriers to low tidal volume ventilation in acute respiratory distress syndrome: survey development, validation, and results. Crit Care Med. 2007 Dec;35(12):2747-54.[ PMID: 17901838]
5.Carlbom DJ, Rubenfeld GD. Barriers to implementing protocol-based sepsis resuscitation in the emergency department--results of a national survey. Crit Care Med. 2007 Nov;35(11):2525-32.[ PMID: 18075366]
Conflict of Interest:
I am a practicing critical care physician.
-
The limitations of standard risk-adjustment methods.
Submit responseTo the editor:
The conclusion that management of critically ill patients by critical care physicians (CCM) is dangerous is premature. The authors state that the results are unexpected and completely contrary to previous findings. The best explanation of this result is the study’s design: the study allows clustering of confounders of mortality which cannot be adjusted by any standard risk-adjustment method (1). Because the study uses few in- and exclusion criteria, not only the patient characteristics in the ‘no-choice’ (CCM) and ‘choice’ group (no CCM) differ significantly in important prognostic factors (e.g., artificial ventilation), but also the ‘treatment environment’ differs (e.g., academic vs. other institutions ). The authors assume that inclusion of SAPS II and propensity-score in the random-effects logistic regression model suffices to correct for these lopsided influences on mortality, which are introduced by treatment choices that lead to severe allocation bias. However, mortality predictions based on scoring systems are known to fall short of a clinician’s overall judgment (2). The estimation of prognosis of a severely ill patient and the decision to place this patient under CCM care, are influenced by more than the sum of variables in the SAPS II and propensity score alone. Thus, an excess of unrecognized influences on mortality is eventually attributed to CCM. To eliminate such treatment allocation bias, some mechanism that approaches randomization is needed (3). For example, Diringer et al., using an early version of the same database, demonstrated that admittance in neurologic/ neurosurgical intensive care units (ICU) improved outcomes in patients with intracerebral hemorrhage (4). By choosing a patient population whose urgency leads to admissions to both specialty and general ICUs, part of the usual allocation bias can be eliminated. In addition, the focus of the analysis by Diringer et al. was not the care giver, but the structure of care. In the study by Levy et al., the argument that subgroup analysis and expansions of the propensity score did not affect the findings of the primary analysis, does not prove robustness, but proves again the limitations of the risk-adjustment method to correct for intractable confounding due to treatment allocation bias (1).
Joris J. Arends, MD Jan P. Vandenbroucke, MD, PhD Leiden University Medical Center NL - 2300 RC
References:
1. Stukel T.A., Fisher E.S., Wennberg D.E. et al.: Analysis of Observational Studies in the Presence of Treatment Selection Bias. JAMA 2007; 297: 278 -285
2. Sinuff T., Adhikari N.K., Cook D.J. et al.: Mortality predictions in the intensive care unit: comparing physicians in scoring systems. Crit Care Med 2006; 34(3): 878-85
3. Vandenbroucke J.P.: When are observational studies as credible as randomized trials? Lancet 2004; 363: 1728-1731
4. Diringer N.M., Edwards D.F.: Admission to a neurologic/ neurosurgical intensive care unit is associated with reduced mortality rate after intracerebral hemorrhage. Crit Care Med 2001; 29: 635 - 640
Conflict of Interest:
None declared
-
Measure of an intensivist : Is mortality rate all that matters?
Submit responseDr. Levy et al published a commendable retrospective analysis in a recent issue of the Annals. Dr. Levy found that the hospital mortality rates were higher for patients who received critical care management (CCM). While the authors insightfully declared several limitations of the study, they did suggest a possibility that CCM might have been associated with poorer outcomes, in particular higher mortality rates. As an intensivist, I would like to interject an opinion on mortality in the intensive care unit (ICU).
Mortality rates have long been the key measures of medical interventions. Unfortunately, as it often happens in the ICU, mortality is an inevitable outcome. When a skilled intensivist realizes that death is imminent, change in direction of care to comfort measures is frequently recommended to the surrogate. This results in decrease ICU and hospital lengths of stay and usually improves the family's satisfaction.
It would have been more meaningful had Dr. Levy and his colleagues enlightened us with comparisons of certain outcome measures, e.g. proportion of patients whose direction of care was changed to comfort measures, ICU and hospital lengths of stay, and family satisfaction with end-of-life care. The value of the intensivist is not only focused on saving lives, but also ensuring that ICU resources are well utilized and our patients are well-cared at the end of their lives.
Respectfully, Petey Laohaburanakit, M.D., F.C.C.P.
Conflict of Interest:
None declared
-
Intensive Caution Required When Interpreting Retrospective Cohort Studies In The Critically Ill.
Submit responsePatients admitted to the ICU are a heterogeneous group of patients, with a wide range of ages, underlying disorders and acute conditions. Given the diversity of this patient population, it is difficult to envision that a single treatment or exposure would show a consistent signal across an entire group of critically ill patients.
The findings of Levy and colleagues showing increased in-hospital mortality rates for those patients treated by critical care physicians compared with non-critical care physicians are therefore surprising(1). Even after adjusting for severity of illness and a propensity score for exposure to critical care, treatment by critical care physicians appears to be associated with increased in-hospital mortality(1).
While the authors made many efforts to collect potential confounders that might affect in-hospital mortality, given the heterogeneous patient population, it is difficult to imagine that one might be able to identify all of the treatment and patient related factors that might be associated with mortality. Of note, a previous attempt to control for severity of illness by using propensity scores in a large heterogeneous cohort of critically ill patients (2) led to results that were discordant with subsequent randomized clinical trials (3-5). Perhaps the most important role of this study by Levy and colleagues will be to stimulate further research into which practices will improve outcomes in critically ill patients.
1. Levy MM, Rapoport J, Lemeshow S, Chalfin DB, Phillips G, Danis M. Association between Critical Care Physician Management and Patient Mortality in the Intensive Care Unit. Ann Intern Med. 2008;148(11):801-9.
2. Connors AF,Jr, Speroff T, Dawson NV, Thomas C, Harrell FE,Jr, Wagner D, et al. The effectiveness of right heart catheterization in the initial care of critically ill patients. SUPPORT Investigators. JAMA. 1996;276(11):889-97.
3. Richard C, Warszawski J, Anguel N, Deye N, Combes A, Barnoud D, et al. Early use of the pulmonary artery catheter and outcomes in patients with shock and acute respiratory distress syndrome: a randomized controlled trial. JAMA. 2003;290(20):2713-20.
4. Binanay C, Califf RM, Hasselblad V, O'Connor CM, Shah MR, Sopko G, et al. Evaluation study of congestive heart failure and pulmonary artery catheterization effectiveness: the ESCAPE trial. JAMA. 2005;294(13):1625- 33.
5. National Heart, Lung, and Blood Institute Acute Respiratory Distress Syndrome (ARDS) Clinical Trials Network, Wheeler AP, Bernard GR, Thompson BT, Schoenfeld D, Wiedemann HP, et al. Pulmonary-artery versus central venous catheter to guide treatment of acute lung injury. N Engl J Med. 2006;354(21):2213-24.
Conflict of Interest:
Dr.Sevransky is a practicing intensivist
-
Unusual major disease categories
Submit responseI was struck by the very unusual major disease categories described under Patient Characteristics in Table 2 of the article.
For example, infections were present in 0-8.1% of the patients in this article. This seems very low.
On the other hand, 19.4-52.9% of patients had a circulatory system disorder, while 12-17.6% had a digestive system and 9.2-19.1% a nervous system disorder. These latter three disease categories were not often the reasons for ICU admission in the mixed medical-surgical ICUs where I worked in the last 6- 7 years.
Particularly the very high proportion of patients with circulatory system disorders and the low frequency of infections suggests to me that perhaps many of the ICUs in this article are in fact coronary care units, or that many of the patients who presented with septic shock, but had no obvious source of infection on admission to the ICU, were misclassified as having a "circulatory system disorder" rather than an infection as the main disease category.
Conflict of Interest:
I am a board-certified pulmonary/critical care physician who worked >6 years as an intensivist in mixed medical-surgical ICUs.
-
In Response
Submit responseTo the Editor:
Both the accompanying editorial and the discussion of their findings provide multiple possible explanations to elucidate the association between intensivists’ care and mortality. Other organizational aspects of the various ICU models and healthcare systems or hospitals studied may not be captured by Project IMPACT and can influence outcome(1).
Regardless, considering the experience of working solely in the Intensive Care Unit, although hospital mortality is an important outcome parameter for some, this is not true for all ICU patients. Taken to its logical extreme, 0% mortality by this analysis under any staffing model would represent the highest performing Intensive Care Unit. Although many would like to believe otherwise, for patients who are going to die, medical intervention prolongs suffering which may be greatest in the Intensive Care Unit(2,3), and adds cost to a healthcare system that is already floundering under the weight of 40 million uninsured Americans, many of whom cannot receive basic healthcare that would clearly improve the quality of their lives. Recognizing that patients can, and actually will, die at the end of their lives, and facilitating this process with a minimum of suffering on the part of both patients and families, while minimizing the cost to the healthcare system is far more important than discharge from the hospital alive. The prolonged suffering and costs incumbent upon some ICU survivors, many of whom never regain health, independent function, or return home could be mitigated by focusing on more important goals for patients and our healthcare system than hospital mortality(4).
Neonatal ICUs in the United States have at least vocalized concern around this issue(5). Perhaps it is time for adult intensivists to do the same.
REFERENCES:
1. Teres D, Higgins T, Steingrub J, Loiacono L, McGee W, Circeo L, et al. Defining a high-performance ICU system for the 21st century: a position paper. J Intensive Care Med. 1998;13:195-205.
2. von Gunten CF, Ferris FD, Emanuel LL. The patient-physician relationship. Ensuring competency in end-of-life care: communication and relational skills. JAMA. 2001;285:1152-1153.
3. Brody H, Campbell ML, Faber-Langendoen K, Ogle KS. Withdrawing intensive life sustaining treatment-recommendations for compassionate clinical management. N Engl J Med. 1997;336(9):652-657.
4. Angus DC, Musthafa AA, Clermont G, Griffin MF, Linde-Zwirble WT,Dremsizov TT, et al. Quality-adjusted survival in the first year after the acute respiratory distress syndrome. Am J Respir Crit Care Med. 2001;163:1389-1394.
5. Eichenwald E, Stark A. Review article: Management and outcomes of very low birth weight. N Engl J Med. 2008;358:1700-1711.
Conflict of Interest:
None declared
-
The trouble with normal
Submit responseWhile selection bias clealry must be considered, in 1996 Connors et al performed a well designed observational study showing that Swan Ganz catheters, previously considered clearly beneficial, were not, and that their use may was harmful. (1) Denials of the study results were immediate and sutained but randomized trials have since resolved the dispute.(2)
Since that time other previously held and intuitively obvious beliefs about normalizing values in critically ill patients have been disproven (3,4).
It would be useful to know how often disproven or unproven therapies are utlized in the different groups. It seems possible that the difference in survival could arise because trained intensivists take greater efforts to normalize parameters in critically ill patients, efforts that could ultimately be causing harm.
1. Connors AF Jr, Speroff T, Dawson NV. The effectiveness of right heart catheterization in the initial care of critically ill patients. SUPPORT Investigators. JAMA. Sep 18 1996;276(11):889-97
2. Harvey S, Harrison DA, Singer M, Ashcroft J, Jones CM, Elbourne D. Assessment of the clinical effectiveness of pulmonary artery catheters in management of patients in intensive care (PAC-Man): a randomised controlled trial. Lancet. Aug 6-12 2005;366(9484):472-7
3. The Albumin Reviewers (Alderson P, Bunn F, Li Wan Po A, Li L, Roberts I, Schierhout G). Human albumin solution for resuscitation and volume expansion in critically ill patients. Cochrane Database of Systematic Reviews 2004, Issue 3.
4. Hébert PC, Wells G, Blajchman MA, et al. A multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. N Engl J Med 1999;340:409-417.
Conflict of Interest:
None declared
-
Association between Critical Care Physician Management and Patient Mortality in the ICU
Submit responseThis article by Levy et al suggests , but does not confirm, that intensive care provided by critical care specialists does not improve outcomes and may be detrimental to patients.(1) The authors have examined data from Project Impact, including 101,832 patients from 123 ICUs in 100 US hospitals. The primary results show that patients cared for by critical care physicians were sicker than other patients and underwent more procedures. Notably, hospital mortality was higher for patients managed by critical care physicians, even after adjustment for severity of illness and other factors that may have influenced the probability of being cared for by a critical care physician. This study is observational in nature, making cause-and-effect conclusions impossible. However, the authors conducted elegant analyses to adjust for other factors that may have influenced survival differently between the critical care and non-critical care groups. Unfortunately, even the most powerful statistical analyses are incapable of dissecting and adjusting for all relevant factors that may influence the outcome of interest. Thus, while observational data such as these may suggest a relationship between intensivist-led patient care and outcomes, ascertaining the real relationship (i.e. cause and effect) requires the conduct of a prospective, randomized trial.
References:
1.Mitchell M. Levy, John Rapoport, Stanley Lemeshow, Donald B. Chalfin, Gary Phillips, and Marion Danis Association between Critical Care Physician Management and Patient Mortality in the Intensive Care Unit Ann Intern Med 2008; 148: 801-809
Conflict of Interest:
None declared
-
Critical Care, Science and Ideology
Submit responseThe study by Levy and colleagues (1) suggests that critically patients cared for by critical care physicians’ (CCP) have a higher mortality rate than those cared for by non-specialists. This observation challenges the “science” (2,3) that has driven public policy for the past decade. Despite application of statistical methods to adjust for sicker patients in the treatment group, it is quite possible that residual confounding accounts for the observed result. While the growing momentum of the critical care movement (3) may entice us to this conclusion, Levy’s findings deserve very careful consideration. The implications for public health are significant.
Let’s assume the observed result is not due to error. The “key exposure was whether the patient was managed by a critical care physician/team.” (1) Critical care outcomes research has suffered from varying, and sometimes arbitrary, definitions. The crux of the problem is the distinction between “intensivist” and critical care trained/certified physician. Notice that even the accompanying editorial, written by world leaders in critical care, conflates the two (4). An intensivist is a physician, usually with training and/or certification in critical care, who supervises or conducts, in person, care of critically ill patients. In 1997, we reported a 26% reduction in mortality (from excessive to acuity-appropriate) associated with adding an intensivist to our hospital (5). The intensivist supervised resident-trainees, often at the bedside during weekdays, 6 months/year, and actuated evidence-based protocols and educational programs. We stress the specificity of this intervention because it is not the same as others – so combining study results in this area may be perilous (2,3). Caregivers in the pre-intensivist period had Board certification, but were not intensivists. They all conducted simultaneously busy outpatient and wards practices. If this (and other) observations (2) were not spurious, it is not the training and certification that impacts outcomes. Rather it is the rapid bedside (and/or telephonic) response of a knowledgeable clinician that likely impacts outcome. It makes perfect sense that even the most knowledgeable critical care physician cannot resuscitate an extremely ill patient as well from the office as she could at the bedside. Patients so close to death cannot afford even small errors. “Being there,” the independent variable at the foundation of intensive care, was not measured in Levy’s study. Another potentially powerful confounder, not considered in statistical adjustments, was the role of trainees in the critical care specialist group. Table 1 suggests that patients receiving CCP care were far more likely to be at teaching hospitals (1). At most such hospitals, trainees provide initial care, much of the day, albeit with CCP guidance. Previous data suggest an association of trainee-provided-critical care and negative outcomes (6,7).
Finally, mortality and disposition are not the only meaningful outcome measures of critical care. To the extent that intensivists are best trained and situated to detect when ongoing care is unlikely to prolong life that is consistent with the values of the patient, and actuate palliative care when appropriate, facilitating death with dignity is an equally important goal of intensivists. Higher mortality is not necessarily a worse outcome.
We urge readers not to discard Levy’s observations because they do not conform with current ideology (3). There are certainly lessons for day-to-day patient care and implications for past and future critical care scientific inquiry.
REFERENCES
1. Levy MM, Rapoport J, Lemeshow S, Chalfin DB, Phillips G, Danis M. Association between critical care physician management and patient mortality in the intensive care unit. Ann Intern Med 2008; 148: 801-809.
2. Pronovost PJ, Angus DC, Dorman T, Robinson KA, Dremsizov TT, Young TL. Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review. JAMA 2002; 288(17): 2151-62.
3. Manthous CA. Leapfrog and critical care: evidence- and reality- based intensive care for the 21st century. Am J Med 2004; 116(3): 188-93.
4. Rubenfeld GD, Angus DC. Are intensivists safe? Ann Inter Med 2008; 148: 877-878.
5. Manthous CA, Amoateng-Adjepong Y, al-Kharrat T, Jacob B, Alnuaimat HM, Chatila W, Hall JB. Effects of a medical intensivist on patient care in a community teaching hospital. Mayo Clin Proc 1997; 72(5): 391-9.
6. Pollack MM, Cuerdon TT, Patel KM, Ruttimann UE, Getson PR, Levetown M. Impact of quality-of-care factors on pediatric intensive care unit mortality. JAMA 1994; 28: 941-6.
7. Landrigan CP, Rothschild JM, Cronin JW, Kaushal R, Burdick E, Katz JT, et al. Effect of reducing interns’ work hours on serious medical errors in intensive care. N Engl J Med 2004; 351: 1838-48.
Conflict of Interest:
None declared
-
Association between Critical Care Physician Management and Patient Mortality
Submit responseTitle: Association between Critical Care Physician Management and Patient Mortality
We read the paper by Levy and colleagues with much interest.[1] We believe that the Project Impact (PI) database was not designed (and is not capable) of answering the question posed and as such it is most likely that the results are erroneous The PI database demonstrates that sicker patients are more likely to be managed in larger academic hospitals by critical care physicians and are more likely to die (because they are sicker!). It is likely that the statistical manipulation of the data could not correct for this association. The critical reader should be extremely suspicious of any observational study using conglomerate propensity scores that assert associations, let alone causality with outcomes. The Connor’s paper, which suggested that pulmonary artery catheters were associated with an increased mortality, provides a key example of this point.[2] The study by Levy and colleagues suggest that critical care physicians are harmful, however, an analysis of their data would equally suggest that academic hospitals as well as hospitals with primary fellowship training programs are harmful.
Current evidence suggests that the organizational structure of an intensive care unit (ICU) has an enormous impact on the quality of care delivered and patient outcome.[3,4] Open units are those in which admission of patients to the ICU is uncontrolled and management of the patients is at the discretion of each attending physician. Admissions are based on a first-come, first-served basis. As the attending of record, usually not a critical care physician, frequently does not have the time nor skills to provide comprehensive critical care he/she "portions off" the patients' care to a number of organ-specific sub-specialists. Whilst this may have some merit in a small number of complex patients, management by proxy or by “committee” frequently results in conflicting treatment strategies. Furthermore, both accountability and responsibility are also apportioned-off, with no physician assuming ultimate responsibility for the patients' care. Such a system is highly cost inefficient and not conducive to achieving optimal patient care. This is the typical organizational structure of most ICUs in the USA. It is likely that the 21 “no choice, non-intensivist” ICUs which participated in PI are not representative of the typical open ICU in the USA. The mere fact that these ICUs participated in PI suggest that they were well organized, evidence-based units with a desire to measure (and improve) their outcomes.
It is also important to recognize that not all “intensivists” (in the USA at least) are created equal. Whilst in Australia, New Zealand and many European counties critical care physicians are solely dedicated to the practice of critical care medicine, this is not so in the USA where many “intensivists” devote less than 50% of their clinical time to the management of patients in the ICU. In addition, in many academic medical centers, particularly Ivy League Institutions, research activities take the “intensivist” out of the ICU for much of their time. It is therefore not uncommon for many “intensivists” to spend as little as 2-4 weeks/year in the ICU. Clearly, unless your professional time is devoted to your specialty you cannot be good at what you do. The American College of Critical Care Medicine defines an “intensivist” as a clinician who has undergone specialty training in critical care medicine, is accredited by the appropriate governing agency and devotes greater than 50% of his/her professional time to the practice of critical care medicine.[5] It is therefore likely that (in the US at least) the type of “intensivist” may influence outcome. This observation is not restricted to critical care medicine. Surgeons and interventional cardiologists who have high volumes have better outcomes than those with low volumes.[6-8] Unfortunately, this study did not provide any information on “full time” intensivists providing “full-time” care.
We assert that the preponderance of evidence supports the concept that critically ill patients are best managed by dedicated critical care physicians who have undergone specialized multidisciplinary training which provides them with the necessary knowledge, skills and attitudes to achieve the best outcomes for critically ill patients. In critical care, it is the intensive attention at the bedside by dedicated physicians and nurses working together in a compassionate and caring environment that will achieve the best outcome for our patients.
Reference List
1. Levy MM, Rapoport J, Lemeshow S et al. Association between critical care physician management and patient mortality iin the Intensive Care Unit. Ann Intern Med 2008; 148:801-809.
2. Connors AF, Speroff T, Dawson NV et al. The effectiveness of right heart catheterization in the initial care of critically ill patients. JAMA 1996; 276:889-897.
3. Pronovost PJ, Angus DC, Dorman T et al. Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review. JAMA 2002; 288:2151-2162.
4. Ghorra S, Reinert SE, Buczko G et al. Analysis of the effect of conversion from open to closed surgical intensive care unit. Ann Surg 1999; 229:163-171.
5. Guidelines for the definition of an intensivist and the practice of critical care. Crit Care Med 1992; 20:540-542.
6. Hogan AM, Winter DC, Hogan AM et al. Does practice make perfect? Ann Surg Oncol 2008; 15:1267-1270.
7. Billingsley KG, Morris AM, Dominitz JA et al. Surgeon and hospital characteristics as predictors of major adverse outcomes following colon cancer surgery: understanding the volume-outcome relationship. Arch Surg 2007; 142:23-31.
8. Magid DJ, Calonge BN, Rumsfeld JS et al. Relation between hospital primary angioplasty volume and mortality for patients with acute MI treated with primary angioplasty vs thrombolytic therapy. JAMA 2000; 284:3131-3138.
Conflict of Interest:
None declared
-
Who follows guidelines?
Submit responseLevy et al suggest that one reason for their unexpected finding of increased mortality in critically ill patients treated by intensivists rather than non- intensivists may be that "... critical care physicians may use their own judgment to manage patients instead of using standardized protocols that may be associated with better outcomes." This statement exposes the authors' bias rather than sheds light on their observation.
When a patient is treated by his or her own physician, that doctor is more likely to customize treatment, based on a deep understanding of the individual, than is a shift-working intensivist who has never laid eyes on the patient.
I was unable to find a report of a trial where patients were randomized to care by their own doctors using medical judgment vs. care by intensivists whose treatment was dictated by guidelines. As Sherlock Holmes said, "It is a capital mistake to theorize in advance of the facts." (Doyle, AC. Adventure of the Second Stain)
Conflict of Interest:
None declared
-
Not time to leap
Submit responseIn their study Levy et al. have reported that, based on the data from Project IMPACT, patients managed by intensivists have a higher risk of hospital death than those managed by a non-critical care specialist [1]. While the authors tried earnestly to adjust away some of the confounding and heterogeneity by applying adjustments for propensity scores and other variables, they have rightfully acknowledged the possibility of residual confounding. However, there are other limitations, some acknowledged and others not, that impair our interpretation of the results.
First, the high number of excluded cases due to missing data predisposes the study to a selection bias. Second, it is our understanding that the authors were unable to ascertain which individual patients were cared for by an intensivist, but rather were forced to divide patients into groups based on the likelihood of being cared for by one. This classification was quite imprecise, resulting in over 50% of patients in each group being lumped into the 5%-95% group, an admittedly vast range of probabilities, predisposing the study to an ecological fallacy. Third, the authors did not stratify their results by “do not resuscitate” status, a factor likely to produce effect heterogeneity. This is an important consideration, since intensivists may be more attentive to end-of-life care practices than non-critical care doctors. Fourth, the endpoint of hospital mortality was not adjusted for exposure time (i.e., length of stay), and thus leaves open the possibility that this outcome was influenced by differing discharge practices between the groups of physicians compared [2].
The current data are at odds with much literature that generally relates outcomes of care to volume, and specifically demonstrates the value of being cared for by an ICU specialist in the setting of critical illness [3]. Because of the vast limitations of the data and the analyses, no firm conclusions can or should be drawn and no “leaps” towards policy change taken based on these findings.
[1] Levy MM, Rapoport J, Lemeshow S, Chalfin DB, Phillips G, Danis M. Association between critical care physician management and patient mortality in the Intensive Care Unit. Ann Intern Med 2008;148:801-809 [2] Kahn JM, Kramer AA, Rubenfeld GD. Transferring critically ill patients out of hospital improves the standardized mortality ratio: a simulation study. Chest 2007;131:68-75 [3] Pronovost PJ, Angus DC, Dorman T, Robinson KA, Dremsizov TT, Young TL. Physician staffing patterns and clinical outcomes in critically ill patients: a systematic review. JAMA. 2002;288:2151-62
Conflict of Interest:
None declared