Multivariate stochastic approximation using a simultaneous perturbation gradient approximation
- 1 March 1992
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Automatic Control
- Vol. 37 (3), 332-341
- https://doi.org/10.1109/9.119632
Abstract
The problem of finding a root of the multivariate gradient equation that arises in function minimization is considered. When only noisy measurements of the function are available, a stochastic approximation (SA) algorithm for the general Kiefer-Wolfowitz type is appropriate for estimating the root. The paper presents an SA algorithm that is based on a simultaneous perturbation gradient approximation instead of the standard finite-difference approximation of Keifer-Wolfowitz type procedures. Theory and numerical experience indicate that the algorithm can be significantly more efficient than the standard algorithms in large-dimensional problems.This publication has 24 references indexed in Scilit:
- On using perturbation analysis to do sensitivity analysis: derivatives vs. differencesPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- A stochastic approximation algorithm for large-dimensional systems in the Kiefer-Wolfowitz settingPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Stochastic approximation for neural network weight estimation in the control of uncertain nonlinear systemsPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2003
- Experiments in nonconvex optimization: Stochastic approximation with function smoothing and simulated annealingNeural Networks, 1990
- A stochastic steepest-descent algorithmJournal of Optimization Theory and Applications, 1988
- On the choice of step size in the Robbins-Monro procedureStatistics & Probability Letters, 1988
- Bayesian error isolation for models of large-scale systemsIEEE Transactions on Automatic Control, 1988
- Empirical Bayes Estimation of Rates in Longitudinal StudiesJournal of the American Statistical Association, 1983
- Asymptotically Efficient Stochastic Approximation; The RM CaseThe Annals of Statistics, 1973
- Accelerated Stochastic ApproximationThe Annals of Mathematical Statistics, 1958