Effects of sample size on classifier design: quadratic and neural network classifiers

Heang-Ping Chan; Berkman Sahiner; Robert F. Wagner; Nicholas Petrick; Joseph T. Mossoba

doi:10.1117/12.274091

25 April 1997 Effects of sample size on classifier design: quadratic and neural network classifiers

Heang-Ping Chan, Berkman Sahiner, Robert F. Wagner, Nicholas Petrick, Joseph T. Mossoba

Author Affiliations +

Proceedings Volume 3034, Medical Imaging 1997: Image Processing; (1997) https://doi.org/10.1117/12.274091
Event: Medical Imaging 1997, 1997, Newport Beach, CA, United States

Abstract

Classifier design is one of the important steps in the development of computer-aided diagnosis (CAD) programs. In this study, we performed simulation studies to evaluate the dependence of the classifier performance on the design sample size, feature space dimensionality, and classifier complexity. The performance of a classifier is quantified by the area (A_z) under the receiver operating characteristic (ROC) curve. Two types of non-linear classifiers, the quadratic discriminants and the backpropagation neural networks, were examined and their performances were compared to those of the linear discriminant classifiers under similar input conditions. A feature space with multivariate normal distributions for the two classes of feature vectors was assumed. A finite sample (N_t) of the normal and abnormal classes was randomly drawn form the populations. A modified cross-validation resampling scheme was used to design the classifiers. By randomly partitioning the available sample set into a training and a test set, a classifier was trained with the design samples and its performance was evaluated by the resubstitution technique and also by testing with the independent test set. For a finite design sample size, it was found that the classifier performance was biased optimistically by resubstitution and pessimistically by testing with the independent set. When the design sample set is sufficiently large, the A_z-versus-1/N_t relationship is approximately linear. The range of N_t in which the linear approximation holds depends on the classifier, the dimensionality of the feature space, and the feature distributions. We analyzed the A_z-versus-1/N_t relationship under a variety of input conditions. The study provides useful information for the design of classifiers in the development of CAD algorithms and other classification problems.

Citation Download Citation

Heang-Ping Chan, Berkman Sahiner, Robert F. Wagner, Nicholas Petrick, and Joseph T. Mossoba "Effects of sample size on classifier design: quadratic and neural network classifiers", Proc. SPIE 3034, Medical Imaging 1997: Image Processing, (25 April 1997); https://doi.org/10.1117/12.274091

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE