Finite-sample effects and resampling plans: applications to linear classifiers in computer-aided diagnosis

Robert F. Wagner; Heang-Ping Chan; Berkman Sahiner; Nicholas Petrick; Joseph T. Mossoba

doi:10.1117/12.274133

25 April 1997 Finite-sample effects and resampling plans: applications to linear classifiers in computer-aided diagnosis

Robert F. Wagner, Heang-Ping Chan, Berkman Sahiner, Nicholas Petrick, Joseph T. Mossoba

Author Affiliations +

Proceedings Volume 3034, Medical Imaging 1997: Image Processing; (1997) https://doi.org/10.1117/12.274133
Event: Medical Imaging 1997, 1997, Newport Beach, CA, United States

Abstract

This work provides an application and extension of the analysis of the effect of finite-sample training and test sets on the bias and variance of the classical discriminants as given by Fukunaga. The extension includes new results for the area under the ROC curve, A_z. An upper bound on A_z is provided by the so-called resubstitution method in which the classifier is trained and tested on the same patients; a lower bound is provided by the hold-out method in which the patient pool is partitioned into trainers and testers. Both methods exhibit a bias in A_z with a linear dependence on the inverse of the number of patients N_t used to train the classifier; this leads to the possibility of obtaining an unbiased estimate of the infinite-population performance by a simple regression procedure. We examine the uncertainties in the resulting estimates. Whereas the bias of classifier performance is determined by the finite size of the training sample, the variance is dominated by the finite size of the test sample. This variance is approximately given by the simple result for an equivalent binomial process. A number of applications to the linear classifier are presented in this paper. More general applications, including the quadratic classifier and some elementary neural-network classifiers, are presented in a companion paper.

Citation Download Citation

Robert F. Wagner, Heang-Ping Chan, Berkman Sahiner, Nicholas Petrick, and Joseph T. Mossoba "Finite-sample effects and resampling plans: applications to linear classifiers in computer-aided diagnosis", Proc. SPIE 3034, Medical Imaging 1997: Image Processing, (25 April 1997); https://doi.org/10.1117/12.274133

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available