How robust are deep object detectors to variability in ground truth bounding boxes? Experiments for target recognition in infrared imagery

Evan A. Stump; Francisco Reveriano; Leslie M. Collins; Jordan M. Malof

doi:10.1117/12.2565897

24 April 2020 How robust are deep object detectors to variability in ground truth bounding boxes? Experiments for target recognition in infrared imagery

Evan A. Stump, Francisco Reveriano, Leslie M. Collins, Jordan M. Malof

Author Affiliations +

Proceedings Volume 11394, Automatic Target Recognition XXX; 1139418 (2020) https://doi.org/10.1117/12.2565897
Event: SPIE Defense + Commercial Sensing, 2020, Online Only

Abstract

In this work we consider the problem of developing deep learning models – such as convolutional neural networks (CNNs) - for automatic target detection (ATD) in infrared (IR) imagery. CNN-based ATD systems must be trained to recognize objects using bounding box (BB) annotations generated by human annotators. We hypothesize that individual annotators may exhibit different biases and/or variability in the characteristics of their BB annotations. Similarly, computer-aided annotation methods may also introduce different types of variability into the BBs. In this work we investigate the impact of BB variability on the behavior and detection performance of CNNs trained using them. We consider two specific BB characteristics here: the center-point, and the overall scale of BBs (with respect to the visual extent of the targets they label). We systematically vary the bias or variance of these characteristics within a large training dataset of IR imagery, and then evaluate the performance on the resulting trained CNN models. Our results indicate that biases in these BB characteristics do not impact performance, but will cause the CNN to mirror the biases in its BB predictions. In contrast, variance in these BB characteristics substantially degrades performance, suggesting care should be taken to reduce variance in the BBs.

Conference Presentation

Citation Download Citation

Evan A. Stump, Francisco Reveriano, Leslie M. Collins, and Jordan M. Malof "How robust are deep object detectors to variability in ground truth bounding boxes? Experiments for target recognition in infrared imagery", Proc. SPIE 11394, Automatic Target Recognition XXX, 1139418 (24 April 2020); https://doi.org/10.1117/12.2565897

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available