RoPE-BAM: nested entity recognition based on rotary position embedding and biaffine attention mechanism

Jinxin Deng; Xizhong Qin; Rong Yang; Xiaoyi Lv

doi:10.1117/12.3004825

9 October 2023 RoPE-BAM: nested entity recognition based on rotary position embedding and biaffine attention mechanism

Jinxin Deng, Xizhong Qin, Rong Yang, Xiaoyi Lv

Proceedings Volume 12791, Third International Conference on Advanced Algorithms and Neural Networks (AANN 2023); 127910Z (2023) https://doi.org/10.1117/12.3004825
Event: Third International Conference on Advanced Algorithms and Neural Networks (AANN 2023), 2023, Qingdao, SD, China

Abstract

Named entity recognition (NER) involves two main types: nested NER and flat NER. The span-based approach classifies entity types by head-tail pair span representations and can handle nested and flat entities uniformly. However, the span-based approach uses a single feature and ignores the relative position information between head-tail pairs, which affects the precision of entity recognition. Therefore, we propose a nested entity recognition model that combines rotary position embedding and biaffine attention mechanism (RoPE-BAM) to improve the model performance by adding relative position features to the span representations. Concretely, we first obtain the head sequence and tail sequence representations through two feedforward networks. Then, to incorporate the relative position features, rotary position embedding is applied to both head and tail sequences. Finally, we use a biaffine attention mechanism to capture the span representations while generating the relative position information in the span. Extensive experiments were conducted on five widely-used benchmark datasets to demonstrate the effectiveness of our proposed RoPE-BAM model.

(2023) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Jinxin Deng, Xizhong Qin, Rong Yang, and Xiaoyi Lv "RoPE-BAM: nested entity recognition based on rotary position embedding and biaffine attention mechanism", Proc. SPIE 12791, Third International Conference on Advanced Algorithms and Neural Networks (AANN 2023), 127910Z (9 October 2023); https://doi.org/10.1117/12.3004825

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available