Do Quality Improvement Collaboratives Improve Antimicrobial Prophylaxis in Surgical Patients?

IN RESPONSE:

We thank Dr. Liu and colleagues for their comments regarding our recent article. They made several statements about our results that readers less familiar with the article may find misleading. Although both groups showed improvements in many aspects of antimicrobial prophylaxis, differences in the extent of improvement between the 2 groups on any indicator were not significant. Our conclusion is correct in this regard. Our study was designed to examine the absolute difference in change between the 2 groups, not the relative difference in change; thus, the proper statistic to evaluate the effect of participation in the collaborative group for the “all-or-none” indicator shows a 6.3% (95% CI, −7.3% to 19.8%) greater improvement in the active intervention.

We agree that improvement interventions are complex social processes that should be theory-based and explained in context by using evaluation models drawn from social science (1, 2). However, we disagree with the statement that the real value of our study lies in the discovery of the heterogeneity and the opportunity for explaining it. Participants in all randomized studies, be they patients or organizations, exhibit heterogeneity. One of the greatest values of a randomized trial design is the ability to answer the question of effectiveness in the face of heterogeneity, thereby addressing the expectation of improvement for the next patient (or organization) that adopts the evaluated treatment. We agree that case studies of success can be extremely helpful in stimulating quality improvement, but the fact that we do not know which mechanisms worked within which circumstances does not negate the value of the cluster randomized trial as an evaluative methodology.

Much needs to be done to improve the quality of health care delivery, and quality improvement collaboratives have a prominent role. In a recent systematic review of the impact of quality improvement collaboratives, Schouten and colleagues (3) concluded that the evidence of their effectiveness is encouraging but limited; they called for studies with a balance of both process-oriented reports and rigorously controlled designs to understand why some collaboratives succeed while others have little effect on practice. Nevertheless, quality improvement collaboratives can be expensive to implement, and tightening resources require the selection of cost-effective strategies. Data from rigorous evaluations, such as that collected for TRAPE (Trial to Reduce Antimicrobial Prophylaxis Errors), are needed to help organizations select among potential improvement strategies.