Visible to infrared transfer learning as a paradigm for accessible real-time object detection and classification in infrared imagery

Yona Falinie A. Gaus; Neelanjan Bhowmik; Brian K. S. Isaac-Medina; Toby P. Breckon

doi:10.1117/12.2573968

20 September 2020 Visible to infrared transfer learning as a paradigm for accessible real-time object detection and classification in infrared imagery

Yona Falinie A. Gaus, Neelanjan Bhowmik, Brian K. S. Isaac-Medina, Toby P. Breckon

Author Affiliations +

Proceedings Volume 11542, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies IV; 1154205 (2020) https://doi.org/10.1117/12.2573968
Event: SPIE Security + Defence, 2020, Online Only

Abstract

Object detection from infrared-band (thermal) imagery has been a challenging problem for many years. With the advent of deep Convolutional Neural Networks (CNN), the automated detection and classification of objects of interest within the scene has become popularised due to the notable increases in performance over earlier approaches in the field. These advances in CNN approaches are underpinned by the availability of large-scale, annotated image datasets that are typically available for visible-band (RGB) imagery. By contrast, there is a lack of prior work that specifically targets object detection in infrared-band images, owing to limited datasets availability that stems from more the limited availability and access to infrared-band imagery and associated hardware in general. A viable solution to this problem is transfer learning which can enable the use of such CNN techniques within infrared-band (thermal) imagery, by leveraging prior training on visible-band (RGB) image datasets, and then subsequently only requiring a secondary, smaller volume of infrared-band (thermal) imagery for CNN model fine-tuning. This is performed by adopting an existing pre-trained CNN, pre-optimized for generalized object recognition in visible-band (RGB) imagery, and subsequently fine-tuning the resultant model weights towards our specific infrared-band (thermal) imagery domain task. We use of two state-of-art object detectors, Single Shot Detector (SSD) with a VGG-16 CNN backbone pre-trained on the ImageNet dataset, and You-Only-Look-Once (YOLOV3) with a DarkNet-53 CNN backbone pretrained on the MS-COCO dataset to illustrate our visible-band to infrared band transfer learning paradigm. Exemplar results reported over the FLIR Thermal and MultispectralFIR benchmark datasets show that significant improvements in mAP detection performance to f0.804_MsFIR, 0.710_FLIRg for SSD and f0.520_MsFIR, 0.308_FLIRgfor YOLOV3 via the use of transfer learning from initial visible-band based CNN training.

Conference Presentation

Citation Download Citation

Yona Falinie A. Gaus, Neelanjan Bhowmik, Brian K. S. Isaac-Medina, and Toby P. Breckon "Visible to infrared transfer learning as a paradigm for accessible real-time object detection and classification in infrared imagery", Proc. SPIE 11542, Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies IV, 1154205 (20 September 2020); https://doi.org/10.1117/12.2573968

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available