Training feed-forward neural networks using conjugate gradients

James L. Blue; Patrick J. Grother

doi:10.1117/12.130286

1 August 1992 Training feed-forward neural networks using conjugate gradients

James L. Blue, Patrick J. Grother

Proceedings Volume 1661, Machine Vision Applications in Character Recognition and Industrial Inspection; (1992) https://doi.org/10.1117/12.130286
Event: SPIE/IS&T 1992 Symposium on Electronic Imaging: Science and Technology, 1992, San Jose, CA, United States

Abstract

Neural networks for optical character recognition are still being trained using back propagation, even though conjugate gradient methods have been shown to be much faster. Most multilayer perceptron network training results in the literature are obtained for small and unrealistic problems or from data sets that are proprietary and not available for comparison testing. We present results on a large realistic pattern set containing 2000 training and 1434 testing exemplars. Each pattern is composed of 32 Gabor coefficients obtained from a 32 by 32 pixel binary image of a handwritten digit segmented from the NIST Handwriting Image Data Base. These sets are believed to have approximately 1 segmentation errors. Comparative results for Moller''s scaled conjugate gradient method and for standard back propagation are presented for runs on a serial scientific workstation and a highly parallel computer. Typical training on a network with 32 inputs, 32 hidden nodes, and 10 output nodes gives a 98 recognition for the training set and 95 for the test set. Training with conjugate gradients requires fewer than 200 iterations; times are about 20 to 40 minutes on a scientific workstation and 6 minutes on the highly parallel computer. Testing (classification) is done at the rate of 600 to 1600 patterns per second on the scientific workstation and on the highly parallel computer respectively. These results suggest that commercial handwritten character recognition systems with great economic potential are feasible.

Citation Download Citation

James L. Blue and Patrick J. Grother "Training feed-forward neural networks using conjugate gradients", Proc. SPIE 1661, Machine Vision Applications in Character Recognition and Industrial Inspection, (1 August 1992); https://doi.org/10.1117/12.130286

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available