 |
 |
|
Home |
Current Issue |
Past Issues |
In the Clinic |
ACP Journal Club |
CME |
Collections |
Audio/Video |
Mobile |
Subscribe |
Tools |
Help |
ACP Online
|
Rapid Responses to:
|
|
Electronic letters published:
![[Read Rapid Response]](/icons/misc/sectionDOWN.gif){kind=icon}
Clinical sensibility and barriers to KT should be considered prior to impact analysis
|
|
|||
|
Jamie C. Brehaut, PhD Ottawa Health Research Institute, Ottawa Hospital, University of Ottawa, Ian G. Stiell
Send rapid response to journal:
jbrehaut{at}ohri.ca Jamie C. Brehaut, et al.
|
Reilly and Evans (1) have provided a very useful discussion of how to study the impact of clinical prediction/decision rules in actual practice. The majority of such rules have not received such evaluations, and this work will hopefully encourage researchers to begin to fill in this gap. We have three specific comments about the article. First, the authors mention in the introduction that “How frequently these and other prediction rules are being used in clinical practice is not known”. While this is certainly true of the vast majority of rules, a fair bit is now known about the rules developed in Ottawa (2-5). Surveys of nationally representative samples of Canadian emergency physicians have shown that the Ottawa Ankle Rules are known to virtually all and reportedly used by almost 90% of Canadian emergency physicians (2;4), while international surveys shows that this figure is dramatically different for other countries. For example, in the U.S. the vast majority were aware of this rule, but only 31% of emergency physicians reported using it(3). This kind of data has been collected on the Ottawa Knee Rules (2), the Canadian C-Spine Rule (5), and is currently being collected for the Canadian CT-Head Rule. Note that while self-reports of rule use tells us whether the physicians feel they make use of the rule, they cannot tell us how the rule is used; i.e. whether physicians are using the rule correctly, using it in all the circumstances for which it would be appropriate to do so, etc. To address these issues,other techniques must be employed (4). Second, Reilly and Evans stress that clinical sensibility of the rule needs to be considered; any rule that does not make clinical sense will be unlikely to be widely used. Yet it is important to remember that clinical sensibility is to some extent idiosyncratic. Interviewing physicians about any specific rule is almost certain to elicit a wide variety of indicators for which it would make clinical sense, at least for some, to include in the rule. If there is general consensus that inclusion of a non-rule indicator would improve the safety or efficiency of the rule, then we agree that evaluating whether inclusion of the indicator into a modified rule may be warranted, even at the relatively late stage of the impact analysis. However, if this new indicator results in a more effective rule, this new rule must then be re-validated on a sample larger than is typical for most impact analyses. Whenever possible, consultation with physicians on all of the potentially useful indicators should be done prior to the derivation of the rule, and not left until the impact analysis. Even when all appropriate factors have been considered during rule derivation, some physicians during the later impact analysis will inevitably feel that the rule could be made safer, more efficient, or more clinically sensible if some of the other indicators were included. Indeed, we have shown that factors known to add no more predictive value over the rule are still considered by a large proportion of physicians purporting to use a rule (4). We have argued that making it clear what factors were included in the derivation of the rule, but were excluded from the rule because they do not add anything, might help improve the perceived clinical sensibility of a rule, and perhaps the level of appropriate use. Finally, Reilly and Evans correctly argue that addressing typical barriers to knowledge translation is a critical component to assessing impact of a rule. Again, we argue that such issues should be considered well before the stage of impact analysis. For example, as the authors suggest, physicians are often more interested in sensitivity than specifity. Indeed, surveys suggest that many physicians would only consider using a rule that had extremely high sensitivity (2). It is for this reason that the Ottawa rules have typically adopted extremely high sensitivity thresholds. While it is clear that reducing the threshold sensitivity would improve the specificity of the rule, our findings suggest that that increase in specificity would come at the cost of fewer physicians using the rule. In order to increase the chances of producing a rule that becomes widely used, such barriers should be considered well before the impact analysis stage, and ideally very early in the development of the rule. Reference List 1. Reilly B,.Evans JS. Translating clinical research into clinical practice: Impact of using prediction rules to make decisions. Annals of Internal Medicine 2006;144:201-9. 2. Graham ID, Stiell IG, Laupacis A, O'Connor AM, Wells GA. Emergency physicians' attitudes toward and use of clinical decision rules for radiography. Academic Emergency Medicine 1998;5:134-40. 3. Graham ID, Stiell IG, Laupacis A, McAuley L, Howell M, Clancy M et al. Awareness and use of the Ottawa Ankle and Knee Rules in 5 countries: Can publication alone be enough to change practice? Annals of Emergency Medicine 2001;37:259-66. 4. Brehaut JC, Stiell I, Graham I, and Visentin L. Clinical decision rules 'in real world': How a widely disseminated rule is used in everyday practice. Academic Emergency Medicine in press. 2005. 5. Brehaut, JC, Stiell, I., and Graham, I. Will a new clinical decision rule be widely used? The case of the Canadian C-Spine Rule (In Press). Acad Emerg Med . 2005. Conflict of Interest:None declared |
|||
