Automated aerial surveillance and detection of hostile ground events, and the tracking of the perpetrators have become of critical importance in the prevention and control of insurgent uprisings and the global war on terror. Yet a basic understanding of the limitations of sensor system coverage as a function of aerial platform position and attitude is often unavailable to program managers and system administrators.
In an effort to better understand this problem we present some of the design tradeoffs for two applications: 1) a 360° viewing focal-plane array sensor system modeled for low altitude aerostat applications, and 2) a fixed diameter area of constant surveillance modeled for high altitude fixed wing aircraft applications. Ground coverage requirement tradeoffs include the number of sensors, sensor footprint geometry, footprint coverage variability as a function of platform position and attitude, and ground surface modeling. Event location specification includes latitude, longitude, altitude for the pixel centroid and corners, and line-of-sight centroid range.
We have performed an experiment that compares the performance of human observers with that of a robust algorithm for the detection of targets in difficult, nonurban forward-looking infrared imagery. Our purpose was to benchmark the comparison and document performance differences for future algorithm improvement. The scale-insensitive detection algorithm, used as a benchmark by the Night Vision Electronic Sensors Directorate for algorithm evaluation, employed a combination of contrastlike features to locate targets. Detection receiver operating characteristic curves and observer-confidence analyses were used to compare human and algorithmic responses and to gain insight into differences. The test database contained ground targets, in natural clutter, whose detectability, as judged by human observers, ranged from easy to very difficult. In general, as compared with human observers, the algorithm detected most of the same targets, but correlated confidence with correct detections poorly and produced many more false alarms at any useful level of performance. Though characterizing human performance was not the intent of this study, results suggest that previous observational experience was not a strong predictor of human performance, and that combining individual human observations by majority vote significantly reduced false-alarm rates.
A shape-based algorithm, designated SPOT, is applied to the output of ARL’s, contrast-like feature based, FLIR anomaly target detector in an effort to improve clutter rejection. The shape-based algorithm uses a spatial symmetry operator that discriminates man-made structures in imagery. The method uses no predetermined templates or filters, which would be range and/or aspect dependent, but rather generates templates on-the-fly from the input data.
Results explore the application of one such symmetry operator, and present a comparative analysis of target detection performance, based on ROC curves and detection histograms.
A series of experiments have been performed to verify the utility of algorithmic tools for the modeling and analysis of cold-target signatures in synthetic, top-attack, FLIR video sequences. The tools include: MuSES/CREATION for the creation of synthetic imagery with targets, an ARL target detection algorithm to detect imbedded synthetic targets in scenes, and an ARL scoring algorithm, using Receiver-Operating-Characteristic (ROC) curve analysis, to evaluate detector performance. Cold-target detection variability was examined as a function of target emissivity, surrounding clutter type, and target placement in non-obscuring clutter locations. Detector metrics were also individually scored so as to characterize the effect of signature/clutter variations.
Results show that using these tools, a detailed, physically meaningful, target detection analysis is possible and that scenario specific target detectors may be developed by selective choice and/or weighting of detector metrics. However, developing these tools into a reliable predictive capability will require the extension of these results to the modeling and analysis of a large number of data sets configured for a wide range of target and clutter conditions.
Finally, these tools should also be useful for the comparison of competitive detection algorithms by providing well defined, and controllable target detection scenarios, as well as for the training and testing of expert human observers.
A series of experiments are performed to benchmark the performance of a target identification classifier trained on synthetic forward-looking infrared (FLIR) target signatures. Results show that the classifier, when trained on synthetic target signatures and tested on measured, real-world target signatures, can perform as well as when trained on measured target signatures alone. It is also shown that when trained on a combined database of measured plus synthetic target signatures, performance exceeds that when trained on either database alone. Finally, it is shown that within a large, diverse database of signatures there exists a subset of signatures whose trained classifier performance can exceed that achieved using the whole database. These results suggest that for classification applications, synthetic FLIR data can be used when enough measured data is unavailable or cannot be obtained due to expense or unavailability of targets, sensors, or site access.
We have designed an experiment that compares the performance of human observers and a scale-insensitive target detection algorithm that uses pixel level information for the detection of ground targets in passive infrared imagery. The test database contains targets near clutter whose detectability ranged from easy to very difficult. Results indicate that human observers detect more "easy-to-detect" targets, and with far fewer false alarms, than the algorithm. For "difficult-to-detect" targets, human and algorithm detection rates are considerably degraded, and algorithm false alarms excessive. Analysis of detections as a function of observer confidence shows that algorithm confidence attribution does not correspond to human attribution, and does not adequately correlate with correct detections. The best target detection score for any human observer was 84%, as compared to 55% for the algorithm for the same false alarm rate. At 81%, the maximum detection score for the algorithm, the same human observer had 6 false alarms per frame as compared to 29 for the algorithm. Detector ROC curves and observer-confidence analysis benchmarks the algorithm and provides insights into algorithm deficiencies and possible paths to improvement.
The performance of infrared (IR) target identification classifiers, trained on randomly selected subsets of target chips taken from larger databases of either synthetic or measured data, is shown to improve rapidly with increasing subset size. This increase continues until the new data no longer provides additional information, or the classifier can not handle the information, at which point classifier performance levels off. It will also be shown that subsets of data selected with advanced knowledge can significantly outperform randomly selected sets, suggesting that classifier training-sets must be carefully selected if optimal performance is desired. Performance will also be shown to be subject to the quality of data used to train the classifier. Thus while increasing training set size generally improves classifier performance, the level to which the classifier performance can be raised will be shown to depend on the similarity between the training data and testing data. In fact, if the training data to be added to a given set of training data is unlike the testing data, performance will often not improve and may possibly diminish. Having too much data can reduce performance as much as having too little. Our results again demonstrate that an infrared (IR) target-identification classifier, trained on synthetic images of targets and tested on measured images, can perform as well as a classifier trained on measured images alone. We also demonstrate that the combination of the measured and the synthetic image databases can be used to train a classifier whose performance exceeds that of classifiers trained on either database alone. Results suggest that it may be possible to select data subsets from image databases that can optimize target classifiers performance for specific locations and operational scenarios.
We report results that demonstrate that an infrared (IR) target classifier, trained on synthetic-images of targets and tested on real-images, can perform as well as a classifier trained on real-images alone. We also demonstrate that the sum of real and synthetic-image databases can be used to train a classifier whose performance exceeds that of classifiers trained on either database alone. After creating a large database of 80,000 synthetic-images two subset databases of 7,000 and 8,000 images were selected and used to train and test a classifier against two comparably sized, sequestered databases of real-images. Synthetic-image selection was accomplished using classifiers trained on real-images from the sequestered real-image databases. The images were chosen if they were correctly identified for both target and target aspect. Results suggest that subsets of synthetic-images can be chosen to selectively train target classifiers for specific locations and operational scenarios; and that it should be possible to train classifiers on synthetic-images that outperform classifiers trained on real-images alone.
Wide-baseline three-dimensional, stereoscopic imaging is being investigated at ARL as an aid to the separation of targets from clutter during operator-assisted target acquisition. Preliminary experimental results at ARL  indicate that false alarms are decreased and the probability of detection is increased. In this paper we will present a program to produce both infrared and visible band synthetic stereoscopic imagery as well as a methodology to evaluate its usefulness for target detection and clutter rejection. Preliminary images are presented, and qualitatively examined and evaluated.
In an effort to improve the usefulness of computer classifiers for military applications, the U.S. Army Research Laboratory has begun to develop a database of synthetic infrared target chips. Once created, this database will aid in the training and testing of both human and computer classifiers, and will provide a way to train classifiers on targets and clutter environments with little real data available. Results presented below will show that classifier performance trained on synthetic data is improving but is, in general, poorer than when trained on real data, that individual synthetic target models perform much better than other models, providing evidence that better overall performance may yet be achievable, that synthetic data thus far created is highly self-similar and/or to some unknown extent represents real data not included in our database, and that enhanced performance of classifiers trained on small amounts of real data can be achieved by adding limited amounts of synthetic data.
Acoustic sensors can be used to detect, track and identify non-line-of-sight targets passively. Attempts to alter acoustic emissions often result in an undesirable performance degradation. This research project investigates the use of neural networks for differentiating between features extracted from the acoustic signatures of sources. Acoustic data were filtered and digitized using a commercially available analog-digital convertor. The digital data was transformed to the frequency domain for additional processing using the FFT. Narrowband peak detection algorithms were incorporated to select peaks above a user defined SNR. These peaks were then used to generate a set of robust features which relate specifically to target components in varying background conditions. The features were then used as input into a backpropagation neural network. A K-means unsupervised clustering algorithm was used to determine the natural clustering of the observations. Comparisons between a feature set consisting of the normalized amplitudes of the first 250 frequency bins of the power spectrum and a set of 11 harmonically related features were made. Initial results indicate that even though some different target types had a tendency to group in the same clusters, the neural network was able to differentiate the targets. Successful identification of acoustic sources under varying operational conditions with high confidence levels was achieved.