Comparison of three-class classification performance metrics: a case study in breast cancer CAD

Amit C. Patel; Mia K. Markey

doi:10.1117/12.595763

Translator Disclaimer

6 April 2005 Comparison of three-class classification performance metrics: a case study in breast cancer CAD

Amit C. Patel, Mia K. Markey

Author Affiliations +

Proceedings Volume 5749, Medical Imaging 2005: Image Perception, Observer Performance, and Technology Assessment; (2005) https://doi.org/10.1117/12.595763
Event: Medical Imaging, 2005, San Diego, California, United States

Abstract

Receiver Operating Characteristic (ROC) analysis is a widely used method for analyzing the performance of two-class classifiers. Advantages of ROC analysis include the fact that it explicitly considers the tradeoffs in sensitivity and specificity, includes visualization methods, and has clearly interpretable summary metrics. Currently, there does not exist a widely accepted performance method similar to ROC analysis for an N-class classifier (N>2). The purpose of this study was to empirically compare methods that have been proposed to evaluate the performance of N-class classifiers (N>2). These methods are, in one way or another, extensions of ROC analysis. This report focuses on three-class classification performance metrics, but most of the methods can be extended easily for more than three classes. The methods studied were pairwise ROC analysis, Hand and Till M Function (HTM), one-versus-all ROC analysis, a modified HTM, and Mossman's "Three-Way ROC" method. A three-class classification task from breast cancer computer-aided diagnosis (CADx) is taken as an example to illustrate the advantages and disadvantages of the alternative performance metrics.

Citation Download Citation

Amit C. Patel and Mia K. Markey "Comparison of three-class classification performance metrics: a case study in breast cancer CAD", Proc. SPIE 5749, Medical Imaging 2005: Image Perception, Observer Performance, and Technology Assessment, (6 April 2005); https://doi.org/10.1117/12.595763

ACCESS THE FULL ARTICLE