Multi-model imaging detection using a learning feature fusion module

Sihao Gao; Yu Cao; Wenjing Zhang; Qian Dai; Jun Li; Xiaojun Xu

doi:10.1117/12.2614776

15 February 2022 Multi-model imaging detection using a learning feature fusion module

Sihao Gao, Yu Cao, Wenjing Zhang, Qian Dai, Jun Li, Xiaojun Xu

Proceedings Volume 12166, Seventh Asia Pacific Conference on Optics Manufacture and 2021 International Forum of Young Scientists on Advanced Optical Manufacturing (APCOM and YSAOM 2021); 121662C (2022) https://doi.org/10.1117/12.2614776
Event: Seventh Asia Pacific Conference on Optics Manufacture and 2021 International Forum of Young Scientists on Advanced Optical Manufacturing (APCOM and YSAOM 2021), 2021, Hong Kong, Hong Kong

Abstract

In this paper, we design a feature fusion module for multi-model imaging based on deep learning. The fused feature of multi-dimensional images is used for object detection, which can effectively avoid interference caused by complex environments. Feature fusion module consists of a convolution layer and an activation function. It can establish the connection between different images. The fusion rules are obtained through supervised learning. Compared with the traditional target detection structure, it can extract more detailed information from several source images. Feature maps extracted from each image are fused by the feature fusion module to form a new feature map. Such a feature map can be better used for the generation of objects masks and bounding boxes. We capture a series of multi-dimensional images with a flexible multi-model camera. When shooting, multi-dimensional information is simultaneously recorded in an image. Through decoding, multiple images of different types can be obtained, including polarization and spectral images. These images record the multi-dimensional optical characteristics of the object and background. Compared with the traditional single-input color or monochrome image method, the proposed method gets 0.25 of average precision and 0.75 of F1-score, which achieves higher detection accuracy in various natural backgrounds.

Citation Download Citation

Sihao Gao, Yu Cao, Wenjing Zhang, Qian Dai, Jun Li, and Xiaojun Xu "Multi-model imaging detection using a learning feature fusion module", Proc. SPIE 12166, Seventh Asia Pacific Conference on Optics Manufacture and 2021 International Forum of Young Scientists on Advanced Optical Manufacturing (APCOM and YSAOM 2021), 121662C (15 February 2022); https://doi.org/10.1117/12.2614776

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Image fusion

Polarization

Cameras

Convolution

Feature extraction

Image processing

Multimodal imaging

Show All Keywords

Keywords/Phrases

Search In:

Publication Years