Automated Detection of Adverse Events Using Natural Language Processing of Discharge Summaries
Open Access
- 31 August 2005
- journal article
- Published by Oxford University Press (OUP) in Journal of the American Medical Informatics Association
- Vol. 12 (4), 448-457
- https://doi.org/10.1197/jamia.m1794
Abstract
Objective: To determine whether natural language processing (NLP) can effectively detect adverse events defined in the New York Patient Occurrence Reporting and Tracking System (NYPORTS) using discharge summaries. Design: An adverse event detection system for discharge summaries using the NLP system MedLEE was constructed to identify 45 NYPORTS event types. The system was first applied to a random sample of 1,000 manually reviewed charts. The system then processed all inpatient cases with electronic discharge summaries for two years. All system-identified events were reviewed, and performance was compared with traditional reporting. Measurements: System sensitivity, specificity, and predictive value, with manual review serving as the gold standard. Results: The system correctly identified 16 of 65 events in 1,000 charts. Of 57,452 total electronic discharge summaries, the system identified 1,590 events in 1,461 cases, and manual review verified 704 events in 652 cases, resulting in an overall sensitivity of 0.28 (95% confidence interval [CI]: 0.17–0.42), specificity of 0.985 (CI: 0.984–0.986), and positive predictive value of 0.45 (CI: 0.42–0.47) for detecting cases with events and an average specificity of 0.9996 (CI: 0.9996–0.9997) per event type. Traditional event reporting detected 322 events during the period (sensitivity 0.09), of which the system identified 110 as well as 594 additional events missed by traditional methods. Conclusion: NLP is an effective technique for detecting a broad range of adverse events in text documents and outperformed traditional and previous automated adverse event detection methods.Keywords
This publication has 31 references indexed in Scilit:
- Extracting information on pneumonia in infants using natural language processing of radiology reportsJournal of Biomedical Informatics, 2005
- Strategies for detecting adverse drug events among older persons in the ambulatory settingJournal of the American Medical Informatics Association, 2004
- Validation of a Discharge Summary Term Search Method to Detect Adverse EventsJournal of the American Medical Informatics Association, 2004
- Modeling Electronic Discharge Summaries as a Simple Temporal Constraint Satisfaction ProblemJournal of the American Medical Informatics Association, 2004
- Electronically Screening Discharge Summaries for Adverse Medical EventsJournal of the American Medical Informatics Association, 2003
- Detecting Adverse Events Using Information TechnologyJournal of the American Medical Informatics Association, 2003
- Identifying Adverse Drug Events: Development of a Computer-based Monitor and Comparison with Chart Review and Stimulated Voluntary ReportJournal of the American Medical Informatics Association, 1998
- Development and Evaluation of a Computerized Admission Diagnoses Encoding SystemComputers and Biomedical Research, 1996
- Introduction: Human Errors and Error HandlingApplied Psychology, 1994
- A General Natural-language Text Processor for Clinical RadiologyJournal of the American Medical Informatics Association, 1994