A perceptron with a skeletal weight-space

Abstract

A perceptron whose space of interactions can interpolate between spherically constrained and binary-valued synapses is introduced and investigated as an associative neural-network memory. For maximally stable storage, and where the weight-space remains connected, the critical storage capacity, alpha _c, is found to be reduced by a factor determined solely by the geometry of the weight space, and is shown to interpolate, within the replica-symmetric approximation, between alpha_c = 2 (in the Gardner-model limit) and alpha_c = 4/pi . Various comparisons of the synaptic weights with those of the binary perceptron show that such differences as remain between this weight space and that of the true binary perceptron are crucial to obtaining alpha_c >or= 4/pi . Although these differences limit the use of such models in realizing optimal binary networks, they may yet provide worthwhile binary systems by simple weight clipping. Simulation results are presented in support of the theoretical analyses.