An improved multi-head-attention and its application for feature extraction and classification as regards the fishery science and technology literature

FengWei Zhang; YingZe Sun; WenXin Tao; Ning Li; Jing Hu; Xue Yan

doi:10.1117/12.2657206

18 October 2022 An improved multi-head-attention and its application for feature extraction and classification as regards the fishery science and technology literature

FengWei Zhang, YingZe Sun, WenXin Tao, Ning Li, Jing Hu, Xue Yan

Author Affiliations +

Proceedings Volume 12349, International Conference on Agri-Photonics and Smart Agricultural Sensing Technologies (ICASAST 2022); 123490G (2022) https://doi.org/10.1117/12.2657206
Event: International Conference on Agri-Photonics and Smart Agricultural Sensing Technologies (ICASAST 2022), 2022, Zhengzhou, China

Abstract

With the information age coming, the explosion of data is creating opportunities just in the management of the text-documents data in and of itself. Although several information technology is applied in web crawl and information category widely, some vital characteristics of such text classification and natural language processing are not able to obtain directly in specific fields. Therefore, this technical paper propose a multi-head-attention method to classify for Fishery Science and Technology Literature, which aims to help scholars or intelligence agent to obtain the classification results of prepared literature from the database directly. There are mainly three contributions in this paper: (i) the multi-head-attention method and improved training mechanisms is first developed for the text classification research; (ii) a novel compression scheme, referred as PP-MiniLM, based on task-agnostic distillation, pruning and quantization is presented to build the fast, small, accurate classification model; (iii) Furthermore, the experimental results indicate that the prediction accuracy is greatly improved based on the pruning and pooling.

Citation Download Citation

FengWei Zhang, YingZe Sun, WenXin Tao, Ning Li, Jing Hu, and Xue Yan "An improved multi-head-attention and its application for feature extraction and classification as regards the fishery science and technology literature", Proc. SPIE 12349, International Conference on Agri-Photonics and Smart Agricultural Sensing Technologies (ICASAST 2022), 123490G (18 October 2022); https://doi.org/10.1117/12.2657206

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Quantization

Data modeling

Feature extraction

Head

Information technology

Performance modeling

Classification systems

Show All Keywords

Keywords/Phrases

Search In:

Publication Years