Automatic detection and tracking of maritime targets in imagery can greatly increase situation awareness on naval vessels. Various methods for detection and tracking have been proposed so far, both for reasoning as well as for learning approaches. Learning approaches have the promise to outperform reasoning approaches. They typically detect targets in a single frame, followed by a tracking step in order to follow targets over time. However, such approaches are sub-optimal for detection of small or distant objects, because these are hard to distinguish in single frames. We propose a new spatiotemporal learning approach that detects targets directly from a series of frames. This new method is based on a deep learning segmentation model and is now applied to temporal input data. This way, targets are detected based not only on appearance in a single frame, but also on their movement over time. Detection hereby becomes more similar to how it is performed by the human eye: by focusing on structures that move differently compared to their surroundings. The performance of the proposed method is compared to both ground-truth detections and detections of a contrast-based detector that detects targets per frame. We investigate the performance on a variety of infrared video datasets, recorded with static and moving cameras, different types of targets, and different scenes. We show that spatiotemporal detection overall obtains similar to slightly better performance on detection of small objects compared to the state-of-the-art frame-wise detection method, while generalizing better with fewer adjustable parameters, and better clutter reduction.
Imaging systems can be used to obtain situational awareness in maritime situations. Important tools for these systems are automatic detection and tracking of objects in the acquired imagery, in which numerous methods are being developed. When designing a detection or tracking algorithm, its quality should be ensured by a comparison with existing algorithms and/or with a ground truth. Detection and tracking methods are often designed for a specific task, so evaluation with respect to this task is crucial, which demands for different evaluation measures for different tasks. We, therefore, propose a variety of quantitative measures for the performance evaluation of detectors and trackers for a variety of tasks. The proposed measures are a rich set from which an algorithm designer can choose in order to optimally design and assess a detection or tracking algorithm for a specific task. We compare these different evaluation measures by using them to assess detection and tracking quality in different maritime detection and tracking situations, obtained from three real-life infrared video data sets. With the proposed set of evaluation measures, a user is able to quantitatively assess the performance of a detector or tracker, which enables an optimal design for his approach.
Detecting maritime targets with electro-optical (EO) sensors is an active area of research. One current trend is to automate target detection through image processing or computer vision. Automation of target detection will decrease the number of people required for lower-level tasks, which frees capacity for higher-level tasks. A second trend is that the targets of interest are changing; more distributed and smaller targets are of increasing interest. Technological trends enable combined detection and identification of targets through machine learning. These trends and new technologies require a new approach in target detection strategies with specific attention to choosing which sensors and platforms to deploy.
In our current research, we propose a ‘maritime detection framework 2.0’, in which multi-platform sensors are combined with detection algorithms. In this paper, we present a comparison of detection algorithms for EO sensors within our developed framework and quantify the performance of this framework on representative data.
Automatic detection can be performed within the proposed framework in three ways: 1) using existing detectors, such as detectors based on movement or local intensities; 2) using a newly developed detector based on saliency on the scene level; and 3) using a state-of-the-art deep learning method. After detection, false alarms are suppressed using consecutive tracking approaches. The performance of these detection methods is compared by evaluating the detection probability versus the false alarm rate for realistic multi-sensor data.
New types of maritime targets require new target detection strategies. Combining new detection strategies with existing tracking technologies shows potential increase in detection performance of the complete framework.