Inheritance of knowledge in neural networks via symbolic algebra

Samir I. Sayegh M.D.

doi:10.1117/12.140117

1 July 1992 Inheritance of knowledge in neural networks via symbolic algebra

Samir I. Sayegh M.D.

Proceedings Volume 1710, Science of Artificial Neural Networks; (1992) https://doi.org/10.1117/12.140117
Event: Aerospace Sensing, 1992, Orlando, FL, United States

Abstract

The theme is that of `inheritance' and the use of symbolic algebra to implement it. The notion of inheriting knowledge between networks is crucial since training a network can be exceedingly slow. Partial information acquired through long experience (learning epochs) should be `transferrable.' A technique using the notion of `gluing' networks has been pioneered by Alex Waibel of CMU. However, this technique cannot be considered true inheritance since the component networks are trained on similar but different subtasks that are later `concatenated.' The approach presented here is based on the observation that even when the problem is not separable, one can get a reasonable performance out of a 2 layer network. Its training is fast and its only minimum is often below that of many of the local minima of the corresponding multi-layer networks. After training such a net, if one can transfer the knowledge to a multi-layer net, not only would one have saved valuable training time, but one would have avoided many of the local minima associated with the multi-layer network. Training can then proceed with the task of the multi-layer net reduced to improving the performance of the 2-layer net, instead of having to start from scratch. Equations corresponding to this approach are derived. They can be written for specific topologies and solved exactly (toys) or approximately (larger problems) using symbolic algebra.

Citation Download Citation

Samir I. Sayegh M.D. "Inheritance of knowledge in neural networks via symbolic algebra", Proc. SPIE 1710, Science of Artificial Neural Networks, (1 July 1992); https://doi.org/10.1117/12.140117

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available