T-03: Development of an AI Approach for Identifying Adverse Events
Poster Presenter
Danielle Abatemarco
Associate Director, PV Strategy, Planning and Execution
BeiGene USA, Inc. United States
Objectives
The objective of this study was to determine if a novel approach utilizing deep learning could accurately identify adverse events in a methodology requiring no feature engineering and that is a scalable, reliable approach.
Method
A neural network was trained from data received between 2015-2016. The neural network was compared to a dictionary-based annotator’s performance regarding precision, sensitivity, and F1 score for identifying the adverse events in an annotated ground truth of spontaneous or solicit cases.
Results
Our neural network yielded a precision, recall, and F1 score of 76.4, 74.9, and 75.6, respectively, for spontaneous report cases and 76.1 recall for solicited cases. These results represent a 49.8 point increase in F1 performance over the dictionary-based method.
Conclusion
Our methodology demonstrates that a neural network approach can achieve accurate adverse event identification in diverse content sources. Further work is required to measure the benefits of a deployed instance of the system and assess its ability to incorporate organization-specific considerations. Neural networks provide a potentially scalable state-of-the-art solution for accurately identifying adverse events in various source documents and with additional data can be further trained to increase performance and reflect organization-specific adverse event detection rules.
