The performance of improved XLNet on text classification

Chenglong Wang; Fenglei Zhang

doi:10.1117/12.2646785

7 September 2022 The performance of improved XLNet on text classification

Chenglong Wang, Fenglei Zhang

Proceedings Volume 12329, Third International Conference on Artificial Intelligence and Electromechanical Automation (AIEA 2022); 123290O (2022) https://doi.org/10.1117/12.2646785
Event: Third International Conference on Artificial Intelligence and Electromechanical Automation (AIEA 2022), 2022, Changsha, China

Abstract

Text classification is the most common application of Natural Language Processing(NLP), and Transformer models have dominated the field in recent years. Currently, pre-training modeling of text through deep learning methods is a common way of text classification. This paper firstly proposes an improved XLNet text classification model based on the problems of long-term dependence and insufficient contextual semantic expression in previous pre-trained language models, and uses XLNet pre-trained language modeling to represent text as low-dimensional word vectors to obtain sequences. Secondly, the generated word vector sequence is passed into the LSTM network, and the two-way features of the sentence are extracted by the memory unit of the LSTM. On the basis of effectively extracting text features, the Multi-head attention model is used to calculate the multi-angle attention probability, to focus on the important words. Finally, text classification is achieved through a fully connected network. The experimental results show that the accuracy of the model is 0.9457 and the loss rate is 0.3133. Compared with other related models, it has achieved better results.

Citation Download Citation

Chenglong Wang and Fenglei Zhang "The performance of improved XLNet on text classification", Proc. SPIE 12329, Third International Conference on Artificial Intelligence and Electromechanical Automation (AIEA 2022), 123290O (7 September 2022); https://doi.org/10.1117/12.2646785

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available