Topological separation versus weight sharing in neural net optimization

Omid M. Omidvar; Charles L. Wilson

doi:10.1117/12.130853

16 December 1992 Topological separation versus weight sharing in neural net optimization

Omid M. Omidvar, Charles L. Wilson

Proceedings Volume 1766, Neural and Stochastic Methods in Image and Signal Processing; (1992) https://doi.org/10.1117/12.130853
Event: San Diego '92, 1992, San Diego, CA, United States

Abstract

Recent advances in neural networks application development for real life problems have drawn attention to network optimization. Most of the known optimization methods rely heavily on a weight sharing concept for pattern separation and recognition. The shortcoming of the weight sharing method is attributed to a large number of extraneous weights which play a minimal role in pattern separation and recognition. Our experiments have shown that up to 97% of the connections in the network can be eliminated with little or no change in the network performance. Topological separation should be used when the size of the network is large enough to tackle real life problems such as fingerprint classification. Our research has focused on the network topology by changing the number of connections as secondary method of optimization. Our findings so far indicate that for large networks topological separation yields smaller network size which is more suitable for VLSI implementation. Topological separation is based on the error surface and information content of the network. As such it is an economical way of size reduction which leads to overall optimization. The differential pruning of the connections is based on the weight contents rather than number of connections. The training error may vary with the topological dynamics but the correlation between the error surface and recognition rate decreases to a minimum. Topological separation reduces the size of the network by changing its architecture without degrading its performance. The method also results in a network which is considerably smaller in size with a better performance.

Citation Download Citation

Omid M. Omidvar and Charles L. Wilson "Topological separation versus weight sharing in neural net optimization", Proc. SPIE 1766, Neural and Stochastic Methods in Image and Signal Processing, (16 December 1992); https://doi.org/10.1117/12.130853

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
12 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Neural networks

Feature extraction

Supercontinuum generation

Image filtering

Image processing

Process control

Signal processing

Show All Keywords

Keywords/Phrases

Search In:

Publication Years