Adaptive structure feed-forward neural networks using polynomial activation functions

Liying Ma; Khashayar Khorasani

doi:10.1117/12.380560

30 March 2000 Adaptive structure feed-forward neural networks using polynomial activation functions

Liying Ma, Khashayar Khorasani

Proceedings Volume 4055, Applications and Science of Computational Intelligence III; (2000) https://doi.org/10.1117/12.380560
Event: AeroSense 2000, 2000, Orlando, FL, United States

Abstract

In cascade-correlation (CC) and constructive one-hidden- layer networks, structural level adaptation is achieved by incorporating new hidden units with identical activation functions one at a time into the active evolutionary net. Functional level adaptation has not received considerable attention, since selecting the activation functions will increase the search space considerably, and a systematic and a rigorous algorithm for accomplishing the search will be required as well. In this paper, we present a new strategy that is applicable to both the fixed structure as well as the constructive network trainings by using different activation functions having hierarchical degrees of nonlinearities, as the constructive learning of a one- hidden-layer feed-forward neural network (FNN) is progressing. Specifically, the orthonormal Hermite polynomials are used as the activation functions of the hidden units, which have certain interesting properties that are beneficial in network training. Simulation results for several noisy regression problems have revealed that our scheme can produce FNNs that generalize much better than one-hidden-layer constructive FNNs with identical sigmoidal activation functions, in particular as applied to rather complicated problems.

Citation Download Citation

Liying Ma and Khashayar Khorasani "Adaptive structure feed-forward neural networks using polynomial activation functions", Proc. SPIE 4055, Applications and Science of Computational Intelligence III, (30 March 2000); https://doi.org/10.1117/12.380560

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available