ROC components-of-variance analysis of competing classifiers

Sergey V. Beiden; Marcus A. Maloof; Robert F. Wagner

doi:10.1117/12.467176

9 May 2002 ROC components-of-variance analysis of competing classifiers

Sergey V. Beiden, Marcus A. Maloof, Robert F. Wagner

Author Affiliations +

Proceedings Volume 4684, Medical Imaging 2002: Image Processing; (2002) https://doi.org/10.1117/12.467176
Event: Medical Imaging 2002, 2002, San Diego, California, United States

Abstract

In the last decade in the field of diagnostic imaging, the problem of variability of reader skill as well as patient case difficulty has given rise to a multivariate approach to receiver operating characteristic (ROC) analysis. The multivariate approach is the so-called multiple-reader, multiple-case (MRMC) ROC paradigm in which every reader reads every patient case and, where possible, in each of two modalities under comparison. The present paper demonstrates the isomorphism between patient cases, image readers, and imaging modalities in diagnostic imaging and, respectively, test sets, training sets, and competing discriminant algorithms in the field of statistical pattern recognition (SPR). Thus, the MRMC paradigm can be brought directly across from imaging to SPR. Recent MRMC ROC analytical methods are demonstrated in the context of SPR for the task of analyzing the natural components of variance in that problem involving test sets, training sets, and competing discriminants. Monte Carlo trials reported here indicate that the conventional wisdom that the variance of measures of classifier accuracy comes mainly from the finite test set is only true when assessing a single algorithm in a very limited context. In particular, it is generally not true when comparing competing discriminant algorithms; in that case the variance is dominated by the finite training set.

Citation Download Citation

Sergey V. Beiden, Marcus A. Maloof, and Robert F. Wagner "ROC components-of-variance analysis of competing classifiers", Proc. SPIE 4684, Medical Imaging 2002: Image Processing, (9 May 2002); https://doi.org/10.1117/12.467176

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE