Sidescan sonar is increasingly accepted as the sensor of choice for sea minehunting over large areas in shallow water. Automatic Target Recognition (ATR) algorithms are therefore being developed to assist and, in the case of autonomous vehicles, even replace the human operator as the primary recognition agent deciding whether an object in the sonar imagery is a mine or simply benign seafloor clutter. Whether ATR aids or replaces a human operator, a natural benchmark for judging the quality of ATR is the unaided human performance when ATR is not used. The benchmark can help when estimating the performance benefit (or cost) of switching from human to automatic recognition for instance, or when planning how human and machine should best interact in cooperative search operations. This paper reports a human performance study using a large library of real sonar images collected for the development and testing of ATR algorithms. The library features 234 mine-like man-made objects deployed for the purpose, as well as 105 instances of naturally occurring clutter. The human benchmark in this case is the average of ten human subjects expressed in terms of a receiver operating characteristic (ROC) curve. An ATR algorithm for man-made/natural object discrimination is also tested and compared with the human benchmark . The implications of its relative performance for the integration of ATR are considered.
KEYWORDS: Smart sensors, Sensors, Defense and security, Matrices, Promethium, Receivers, Information security, Communication engineering, Information theory, Intelligent sensors
Starting with the supposition that the product of smart sensors - whether autonomous, networked, or fused - is in all cases information, it is shown here using information theory how a metric Q, ranging between 0 and 100%, can be derived to assess the quality of the information provided. The analogy with student grades is immediately evident and elaborated. As with student grades, numerical percentages suggest more precision than can be justified, so a conversion to letter grades A+ to D- is desirable. Owing to the close analogy with familiar academic grades, moreover, the new grade is a measure of effectiveness (MOE) that commanders and decision makers should immediately appreciate and find quite natural, even if they do not care to follow the methodology behind the performance test, as they focus on higher-level strategic matters of sensor deployment or procurement. The metric is illustrated by translating three specialist performance tests - the Receiver Operating Characteristic (ROC) curve, the Constant False Alarm Rate (CFAR) approach, and confusion matrices - into letter grades for use then by strategists. Actual military and security systems are included among the examples.
Remote sensing is valuable for ISR insofar as it provides information relevant to the detection, classification, identification, and tracking of one or more targets of interest. A new system-level measure of the quality of ISR is given here. By combining information and detection theory, the measure gives a quantitative estimate of the relative proportions of reliable (good), misleading (bad), and missing information. Being probabilistic, the measure can be used to assess the quality of day-to-day surveillance, or to estimate the quality expected for more narrowly defined and speculative scenarios. The measure is intended for use in a cost-benefit study of sorts, comparing the quality of different sensors or mixes of sensors for ISR. The quality metric is demonstrated here using a maritime interdiction scenario from maritime ISR.
High-resolution sidescan sonars are often used in underwater warfare for large-area surveys of the seafloor in the search for sea mines. Much effort has gone toward the automatic detection of sea mines. In its more advanced forms, such auto-detection entails pattern recognition: the automatic assignment of class labels (target/non-target) to signatures according to their distinctive features. This paper demonstrates a texture-based feature for automatically discriminating between man-made and natural objects. Real sonar data is used, and the demonstration includes performance estimates in the form of the receiver-operator characteristic (ROC) curves necessary (though often omitted) for evaluating detectors for operational use. The merits of redefining the allowable automatic responses-from the classes of mine targets ultimately sought, to the class of man-made objects more generally-are reviewed from both the pattern-recognition and operational perspectives.
Modern sidescan sonars produce echographs that look rather like aerial photographs of the seafloor, with a few important differences. Not least is their obvious sonar speckle---random pixel-to-pixel variations of the image intensity across flat surfaces (smooth sand, mud, clay, or fine gravel for instance) that cannot be attributed to system noise alone. Speckle is in fact sediment dependent, suggesting that it might be used for sediment classification, but the wide variation of speckle typically encountered within each traditional sediment class must first be overcome. Following a different approach, we use speckle to automatically subdivide the image into just two, much broader classes that are relevant in a search for objects: 1) regions of interest (ROI) where attention is warranted because their pixel-to-pixel variations cannot be attributed to speckle alone---i.e., resolvable seafloor features or distinct objects must be present; and 2) empty regions whose pixel-to-pixel variations are speckle-like and therefore of no interest. The distinction is posed as a hypothesis test based on physical and statistical theory. The test is suited for detecting small targets comprised of few pixels whose intensity and uniformity are unlikely deviations from local speckle statistics.
Needless to say, large, high-contrast targets are more easily recognized than small, low-contrast targets. But at what size and contrast does recognition begin? The question is important when specifying the resolution and contrast requirements for a new imaging system, or when assessing the range of an existing system beyond which worsening resolution and contrast ruin serviceable performance. The question is addressed here, in a general way, under the assumption that recognition depends on the agent's ability to draw a line (extract an edge) around distinctive target features. If the target is small, moreover, with its recognizable facets occupying few pixels, and if line rendering suffers mainly due to noise or image speckle, then neither human or complex automatic target recognition systems have a clear advantage, one above the other, or above more tractable, statistically optimized pattern recognition algorithms. Thus a theory of optimal linear edge detection is proposed here as a plausible model for estimating the recognition limits of both human and automatic agents, making it possible to estimate when the line-rendering process, and hence, recognition, fails due to insufficient contrast for small targets. The method is used to estimate the shadow-background contrasts needed for the recognition of sea mines in sidescan sonar images.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.