 |
 |
|
Home |
Current Issue |
Past Issues |
In the Clinic |
ACP Journal Club |
CME |
Collections |
Audio/Video |
Mobile |
Subscribe |
Tools |
Help |
ACP Online
|
1 May 1995 | Volume 122 Issue 9 | Pages 681-688
Objective: To evaluate the automated detection of clinical conditions described in narrative reports.
Design: Automated methods and human experts detected the presence or absence of six clinical conditions in 200 admission chest radiograph reports.
Study Subjects: A computerized, general-purpose natural language processor; 6 internists; 6 radiologists; 6 lay persons; and 3 other computer methods.
Main Outcome Measures: Intersubject disagreement was quantified by "distance" (the average number of clinical conditions per report on which two subjects disagreed) and by sensitivity and specificity with respect to the physicians.
Results: Using a majority vote, physicians detected 101 conditions in the 200 reports (0.51 per report); the most common condition was acute bacterial pneumonia (prevalence, 0.14), and the least common was chronic obstructive pulmonary disease (prevalence, 0.03). Pairs of physicians disagreed on the presence of at least 1 condition for an average of 20% of reports. The average intersubject distance among physicians was 0.24 (95% CI, 0.19 to 0.29) out of a maximum possible distance of 6. No physician had a significantly greater distance than the average. The average distance of the natural language processor from the physicians was 0.26 (CI, 0.21 to 0.32; not significantly greater than the average among physicians). Lay persons and alternative computer methods had significantly greater distance from the physicians (all >0.5). The natural language processor had a sensitivity of 81% (CI, 73% to 87%) and a specificity of 98% (CI, 97% to 99%); physicians had an average sensitivity of 85% and an average specificity of 98%.
Conclusions: Physicians disagreed on the interpretation of narrative reports, but this was not caused by outlier physicians or a consistent difference in the way internists and radiologists read reports. The natural language processor was not distinguishable from the physicians and was superior to all other comparison subjects. Although the domain of this study was restricted (six clinical conditions in chest radiographs), natural language processing seems to have the potential to extract clinical information from narrative reports in a manner that will support automated decision-support and clinical research.
Author and Article Information
From Columbia-Presbyterian Medical Center, New York, New York and Queens College, Flushing, New York.
ACADEMIA AND CLINIC
Unlocking Clinical Data from Narrative Reports: A Study of Natural Language Processing
Requests for Reprints: George Hripcsak, MD, Department of Medical Informatics, Columbia-Presbyterian Medical Center, 161 Fort Washington Avenue, AP-1310, New York, NY 10032.
Grant Support: National Library of Medicine grants LM04419, LM05397, and LM05627; grant #6-61483 from the Research Foundation of City University of New York.
Related articles in Annals:
This article has been cited by other articles:
|
|
 |
|
  L. Zhou, S. Parsons, and G. Hripcsak The Evaluation of a Temporal Reasoning System in Processing Clinical Discharge Summaries J. Am. Med. Inform. Assoc., January 1, 2008; 15(1): 99 - 106. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Y. A. Lussier and Y. Liu Computational Approaches to Phenotyping: High-Throughput Phenomics Proceedings of the ATS, January 1, 2007; 4(1): 18 - 25. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. H. Thrall Reinventing Radiology in the Digital Age: Part II. New Directions and New Stakeholder Value Radiology, October 1, 2005; 237(1): 15 - 18. [Full Text] [PDF]
|
|
|
|
|
|
B. Hazlehurst, H. R. Frost, D. F. Sittig, and V. J. Stevens MediClass: A System for Detecting and Classifying Encounter-based Clinical Events in Any Electronic Medical Record J. Am. Med. Inform. Assoc., September 1, 2005; 12(5): 517 - 529. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. B. Melton and G. Hripcsak Automated Detection of Adverse Events Using Natural Language Processing of Discharge Summaries J. Am. Med. Inform. Assoc., July 1, 2005; 12(4): 448 - 457. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. R.O. Payne and J. B. Starren Quantifying Visual Similarity in Clinical Iconic Graphics J. Am. Med. Inform. Assoc., May 1, 2005; 12(3): 338 - 345. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. J. Thomas, H. Ouellette, E. F. Halpern, and D. I. Rosenthal Automated Computer-Assisted Categorization of Radiology Reports Am. J. Roentgenol., February 1, 2005; 184(2): 687 - 690. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
T. S. Field, J. H. Gurwitz, L. R. Harrold, J. M. Rothschild, K. Debellis, A. C. Seger, L. S. Fish, L. Garber, M. Kelleher, and D. W. Bates Strategies for Detecting Adverse Drug Events among Older Persons in the Ambulatory Setting J. Am. Med. Inform. Assoc., November 1, 2004; 11(6): 492 - 498. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Friedman, L. Shagina, Y. Lussier, and G. Hripcsak Automated Encoding of Clinical Documents Based on Natural Language Processing J. Am. Med. Inform. Assoc., September 1, 2004; 11(5): 392 - 402. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. Aronsky, E. Kasworm, J. A. Jacobson, P. J. Haug, and N. C. Dean Electronic Screening of Dictated Reports to Identify Patients with Do-Not-Resuscitate Status J. Am. Med. Inform. Assoc., September 1, 2004; 11(5): 403 - 409. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
W. W. Chapman, G. F. Cooper, P. Hanbury, B. E. Chapman, L. H. Harrison, and M. M. Wagner Creating a Text Classifier to Detect Radiology Reports Describing Mediastinal Findings Associated with Inhalational Anthrax and Other Disorders J. Am. Med. Inform. Assoc., September 1, 2003; 10(5): 494 - 503. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. B. Wilcox and G. Hripcsak The Role of Domain Knowledge in Automating Medical Text Report Classification J. Am. Med. Inform. Assoc., July 1, 2003; 10(4): 330 - 338. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. J. Murff, A. J. Forster, J. F. Peterson, J. M. Fiskio, H. L. Heiman, and D. W. Bates Electronically Screening Discharge Summaries for Adverse Medical Events J. Am. Med. Inform. Assoc., July 1, 2003; 10(4): 339 - 350. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. W. Bates, R. S. Evans, H. Murff, P. D. Stetson, L. Pizziferri, and G. Hripcsak Detecting Adverse Events Using Information Technology J. Am. Med. Inform. Assoc., March 1, 2003; 10(2): 115 - 128. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Hripcsak, J. H. M. Austin, P. O. Alderson, and C. Friedman Use of Natural Language Processing to Translate Clinical Information from a Database of 889,921 Chest Radiographic Reports Radiology, July 1, 2002; 224(1): 157 - 163. [Abstract] [Full Text]
|
|
|
|
|
|
H. Yu, G. Hripcsak, and C. Friedman Mapping Abbreviations to Full Forms in Biomedical Articles J. Am. Med. Inform. Assoc., May 1, 2002; 9(3): 262 - 272. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Hripcsak and A. Wilcox Reference Standards, Judges, and Comparison Subjects: Roles for Experts in Evaluating System Performance J. Am. Med. Inform. Assoc., January 1, 2002; 9(1): 1 - 15. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. K. Taira, S. G. Soderland, and R. M. Jakobovits Automatic Structuring of Radiology Free-Text Reports RadioGraphics, January 1, 2001; 21(1): 237 - 245. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Fiszman, W. W. Chapman, D. Aronsky, R. S. Evans, and P. J. Haug Automatic Detection of Acute Bacterial Pneumonia from Chest X-ray Reports J. Am. Med. Inform. Assoc., November 1, 2000; 7(6): 593 - 604. [Abstract] [Full Text]
|
|
|
|
|
|
W. W. Stead, R. A. Miller, M. A. Musen, and W. R. Hersh Integration and Beyond: Linking Information from Disparate Sources andinto Workflow J. Am. Med. Inform. Assoc., March 1, 2000; 7(2): 135 - 145. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Hripcsak, G. J. Kuperman, C. Friedman, and D. F. Heitjan A Reliability Study for Evaluating Information Extraction from Radiology Reports J. Am. Med. Inform. Assoc., March 1, 1999; 6(2): 143 - 150. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Friedman, G. Hripcsak, L. Shagina, and H. Liu Representing Information in Patient Reports Using Natural Language Processing and the Extensible Markup Language J. Am. Med. Inform. Assoc., January 1, 1999; 6(1): 76 - 87. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Hripcsak, C. A. Knirsch, N. L. Jain, and A. Pablos-Mendez Automated Tuberculosis Detection J. Am. Med. Inform. Assoc., September 1, 1997; 4(5): 376 - 381. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 W. Evans Computer-Supported Content Analysis: Trends, Tools, and Techniques Social Science Computer Review, October 1, 1996; 14(3): 269 - 279. [Abstract] [PDF]
|
|
Article
