D. Zaikis, S. Kokkas, I. Vlahavas, "Transforming Drug-Drug Interaction Extraction from Biomedical Literature", In Proceedings of the 12th Hellenic Conference on Artificial Intelligence (SETN '22), Corfu, Greece, 7-9 Sep 2022, Article 12, pp. 1–8, Association for Computing Machinery, 2022
Author(s): Dimitrios Zaikis, Stylianos Kokkas, and Ioannis Vlahavas
Keywords: neural networks, drug-drug interactions, transformers, language models
Tags:
Abstract: Language Models (LM) capture the characteristics of the distribution of words sequences in natural language, learning meaningful distributed representations in the process. Recent advancements in Neural Networks and Deep Leaning have led to rapid progress in this area, greatly attributed to the emergence of the attention mechanism. Specifically, the Transformer architecture that implements attention-based encoder-decoder stacks, has advanced research in numerous Natural Language Processing tasks and produced state-of-the-art pre-trained LMs. One important task is Relationship Extraction (RE), which extracts semantic relationships from text and has significant applications in the biomedical domain, especially in literature pertaining to drug safety and Drug-Drug Interactions (DDI). In DDI extraction, the task is divided into two subtasks, Drug Named Entity Recognition and Relation Classification, consequently identifying drug mentions and classifying the potential effect of drug combinations from literature. Various methods for the extraction of DDIs have been proposed that utilize different architectures, however Transformers-based LMs continue to show the most promise. The overwhelming number of available pre-trained LMs, that each provide their own benefits and disadvantages, renders the selection of a clear baseline extremely difficult. In this paper, we investigate the most relevant LMs for biomedical RE and experiment on the DDI Extraction 2013 dataset. We propose a baseline approach for pre-trained Transformer-based LMs with shallow output architectures to effectively utilize the underlying architecture and introduce a foundation that reaches similar to state-of-the-art performance for both subtasks of DDI extraction.