Statistically controlled activation weight initialization (SCAWI)
- 1 July 1992
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Neural Networks
- Vol. 3 (4), 627-631
- https://doi.org/10.1109/72.143378
Abstract
An optimum weight initialization which strongly improves the performance of the back propagation (BP) algorithm is suggested. By statistical analysis, the scale factor, R (which is proportional to the maximum magnitude of the weights), is obtained as a function of the paralyzed neuron percentage (PNP). Also, by computer simulation, the performances on the convergence speed have been related to PNP. An optimum range for R is shown to exist in order to minimize the time needed to reach the minimum of the cost function. Normalization factors are properly defined, which leads to a distribution of the activations independent of the neurons, and to a single nondimensional quantity, R, the value of which can be quickly found by computer simulationKeywords
This publication has 3 references indexed in Scilit:
- Back-propagation algorithm which varies the number of hidden unitsNeural Networks, 1991
- Complete solution of the local minima in the XOR problemNetwork: Computation in Neural Systems, 1991
- An optimum weights initialization for improving scaling relationships in BP learningPublished by Elsevier BV ,1991