A UK MoD funded programme into autonomous sensors arrays (SAPIENT) has been developing new, highly capable sensor modules together with a scalable modular architecture for control and communication. As part of this system there is a desire to also utilise existing legacy sensors. The paper reports upon the development of a SAPIENT-compliant sensor module using a legacy Close-Circuit Television (CCTV) pan-tilt-zoom (PTZ) camera. The PTZ camera sensor provides three modes of operation. In the first mode, the camera is automatically slewed to acquire imagery of a specified scene area, e.g. to provide “eyes-on” confirmation for a human operator or for forensic purposes. In the second mode, the camera is directed to monitor an area of interest, with zoom level automatically optimized for human detection at the appropriate range. Open source algorithms (using OpenCV) are used to automatically detect pedestrians; their real world positions are estimated and communicated back to the SAPIENT central fusion system. In the third mode of operation a “follow” mode is implemented where the camera maintains the detected person within the camera field-of-view without requiring an end-user to directly control the camera with a joystick.
Currently, most land Intelligence, Surveillance and Reconnaissance (ISR) assets (e.g. EO/IR cameras) are simply data collectors. Understanding, decision making and sensor control are performed by the human operators, involving high cognitive load. Any automation in the system has traditionally involved bespoke design of centralised systems that are highly specific for the assets/targets/environment under consideration, resulting in complex, non-flexible systems that exhibit poor interoperability. We address a concept of Autonomous Sensor Modules (ASMs) for land ISR, where these modules have the ability to make low-level decisions on their own in order to fulfil a higher-level objective, and plug in, with the minimum of preconfiguration, to a High Level Decision Making Module (HLDMM) through a middleware integration layer. The dual requisites of autonomy and interoperability create challenges around information fusion and asset management in an autonomous hierarchical system, which are addressed in this work. This paper presents the results of a demonstration system, known as Sensing for Asset Protection with Integrated Electronic Networked Technology (SAPIENT), which was shown in realistic base protection scenarios with live sensors and targets. The SAPIENT system performed sensor cueing, intelligent fusion, sensor tasking, target hand-off and compensation for compromised sensors, without human control, and enabled rapid integration of ISR assets at the time of system deployment, rather than at design-time. Potential benefits include rapid interoperability for coalition operations, situation understanding with low operator cognitive burden and autonomous sensor management in heterogenous sensor systems.
Performing persistent surveillance of large populations of targets is increasingly important in both the defence and security domains. In response to this, Wide Area Motion Imagery (WAMI) sensors with Wide FoVs are growing in popularity. Such WAMI sensors simultaneously provide high spatial and temporal resolutions, giving extreme pixel counts over large geographical areas. The ensuing data rates are such that either very bandwidth data links are required (e.g. for human interpretation) or close-to-sensor automation is required to down-select salient information. For the latter case, we use an iterative quad-tree optical-flow algorithm to efficiently estimate the parameters of a perspective deformation of the background. We then use a robust estimator to simultaneously detect foreground pixels and infer the parameters of each background pixel in the current image. The resulting detections are referenced to the coordinates of the first frame and passed to a multi-target tracker. The multi-target tracker uses a Kalman filter per target and a Global Nearest Neighbour approach to multi-target data association, thereby including statistical models for missed detections and false alarms. We use spatial data structures to ensure that the tracker can scale to analysing thousands of targets. We demonstrate that real-time processing (on modest hardware) is feasible on an unclassified WAMI infra-red dataset consisting of 4096 by 4096 pixels at 1Hz simulating data taken from a Wide FoV sensor on a UAV. With low latency and despite intermittent obscuration and false alarms, we demonstrate persistent tracking of all but one (low-contrast) vehicular target, with no false tracks.
The report describes the results of a multi-year programme of research aimed at the development of an integrated multi-sensor perimeter detection system capable of being deployed at an operational site. The research was driven by end user requirements in protective security, particularly in threat detection and assessment, where effective capability was either not available or prohibitively expensive. Novel video analytics have been designed to provide robust detection of pedestrians in clutter while new radar detection and tracking algorithms provide wide area day/night surveillance. A modular integrated architecture based on commercially available components has been developed. A graphical user interface allows intuitive interaction and visualisation with the sensors. The fusion of video, radar and other sensor data provides the basis of a threat detection capability for real life conditions. The system was designed to be modular and extendable in order to accommodate future and legacy surveillance sensors. The current sensor mix includes stereoscopic video cameras, mmWave ground movement radar, CCTV and a commercially available perimeter detection cable. The paper outlines the development of the system and describes the lessons learnt after deployment in a pilot trial.
The next generation of military reconnaissance systems will generate larger images, with more bits per pixel, combined with improved spatial, spectral and temporal resolutions. This increase in the quantity of imagery requiring analysis to produce militarily useful reports, the "data deluge", will place a significant burden upon dissemination systems and upon traditional, largely manual, exploitation techniques. To avoid the possibility that imagery derived from expensive assets may go unexploited, an increased use of automated imagery exploitation tools is required to assist the Image Analysts (IAs) in their tasks.
This paper describes the fully automated generation of time sequence datacubes from mixed imagery sources as cueing aids for IAs. In addition to facilitating quick and easy visual inspection, the datacubes provide the prealigned image sets needed for exploitation by some automated change detection and target detection algorithms. The ARACHNID system under development handles SAR, IR and EO imagery and will align image pairs obtained with widely differing sun, view and zenith angles. Edge enhancement pre-processing is employed to increase the similarity between images of disparate characteristics.
Progress is reported on the automation of this registration task, on its current performance characteristics, its potential for integration into an operational system, and on its expected utility in this context.
Increasingly demanding military requirements and rapid technological advances are producing reconnaissance sensors with greater spatial, spectral and temporal resolution. This, with the benefits to be gained from deploying multiple sensors co-operatively, is resulting in a so-called data deluge, where recording systems, data-links, and exploitation systems struggle to cope with the required imagery throughput. This paper focuses on the exploitation stage and, in particular, the provision of cueing aids for Imagery Analysts (IAs), who need to integrate a variety of sources in order to gain situational awareness. These sources may include multi-source imagery and intelligence feeds, various types of mapping and collateral data, as well the need for the IAs to add their own expertise in military doctrine etc. This integration task is becoming increasingly difficult as the volume and diversity of the input increases. The first stage in many exploitation tasks is that of image registration. It facilitates change detection and many avenues of multi-source exploitation. Progress is reported on the automating this task, on its current performance characteristics, its integration into a potentially operational system, and hence on its expected utility. We also report on the development of an evolutionary architecture, 'ICARUS' in which feature detectors (or cuers) are constructed incrementally using a genetic algorithm that evolves simple sub-structures before combining, and further evolving them, to form more comprehensive and robust detectors. This approach is shown to help overcome the complexity limit that prevents many machine-learning algorithms from scaling up to the real world.
It has been known for some time that millimeter waves can pas through clothing. In short range applications such as in the scanning of people for security purposes, operating at Ka band can be an advantage. The penetration through clothing is increased and the cost of the equipment when compared to operation at W band. In this paper a Ka band mechanically scanned imager designed for security scanning is discussed. This imager is based on the folded conical scan technology previously reported. It is constructed from low cost materials such as polystyrene and printed circuit board. The trade off between image spatial resolution and the number of receivers will be described and solutions, which minimize this number discussed.
One of the difficulties that has been apparent in applying image processing algorithms not just for automatic target recognition but also for associated tasks in image processing and understanding is that of the optimal choice of parameters and algorithms. Firstly we must select an algorithm to use and secondly the actual parameters that are required by that algorithm. It is also the case that using a chosen algorithm on a different image class yields results of a totally different quality, here we consider three image classes, namely infra-red linescan, dd5-Russian satellite and SPOT imagery. We are now exploring the use of genetic algorithms for the purpose of parameter and algorithm selection and will show how the approach can successfully obtain results which in the past have tended to be obtained somewhat heuristically.
Progress is reviewed on the development of an all source image interpretation system which exploits complementary evidence from a range of experts. This co-operation may occur between feature detectors in different bands, between detectors searching for different types of feature, or between different types of detector of the same feature. Algorithms for detecting vehicles in infrared linescan imagery gives a low missed detection rate but have been found to respond falsely to: roads fragmented by trees; structures such as cylindrical storage tanks; and to corners of man made objects, such as buildings. False alarms are reduced by applying algorithms which detect subclasses of false alarms reliably i.e. buildings and storage tanks. In addition, both are features of interest in themselves, and are useful primitives in the identification of sites. The integration of depth (in the form of disparity maps) is examined as a means of reducing false building detections. Outputs from the feature detectors are combined using a simple rule-based approach. A surface based model matching technique is examined as a means of classifying the remaining vehicle candidates.