Probability densities and confidence intervals for target recognition performance metrics

David R. Parker; Steven C. Gustafson; Timothy D. Ross

doi:10.1117/12.602322

Translator Disclaimer

19 May 2005 Probability densities and confidence intervals for target recognition performance metrics

David R. Parker, Steven C. Gustafson, Timothy D. Ross

Author Affiliations +

Proceedings Volume 5808, Algorithms for Synthetic Aperture Radar Imagery XII; (2005) https://doi.org/10.1117/12.602322
Event: Defense and Security, 2005, Orlando, Florida, United States

Abstract

Probability densities for target recognition performance metrics are developed. These densities assist in evaluation of systems under test (SUTs), which are systems that predict the presence of a target after examination of an input. After such examination, a SUT assigns a score that indicates the predicted likelihood that a target is present. From scores for a series of many inputs, the suitability of a SUT can be evaluated through performance metrics such as the receiver operating characteristic (ROC) and the confidence error (CE) generation curve. The ROC is a metric that describes how well the probability densities of target and clutter scores are separated, where clutter refers to the absence of target. The CE generation curve and the corresponding scalar CE is a metric that evaluates the accuracy of the score. Since only a limited number of test scores (scores for which the truth state is known by the evaluator) is typically available to evaluate a SUT, it is critical to quantify uncertainty in the performance metric results. A process for estimating such uncertainty through probability densities for the performance metrics is examined here. Once the probability densities are developed, confidence intervals are also obtained. The process that develops the densities and related confidence intervals is implemented in a fully Bayesian manner. Two approaches are examined, one which makes initial assumptions regarding the form of the underlying target and clutter densities and a second approach which avoids such assumptions. The target and clutter density approach is applicable to additional performance metrics.

Citation Download Citation

David R. Parker, Steven C. Gustafson, and Timothy D. Ross "Probability densities and confidence intervals for target recognition performance metrics", Proc. SPIE 5808, Algorithms for Synthetic Aperture Radar Imagery XII, (19 May 2005); https://doi.org/10.1117/12.602322

ACCESS THE FULL ARTICLE