Convergence of stochastic algorithms: from the Kushner–Clark theorem to the Lyapounov functional method

Abstract

In the first part of this paper a global Kushner–Clark theorem about the convergence of stochastic algorithms is proved: we show that, under some natural assumptions, one can ‘read' from the trajectories of its ODE whether or not an algorithm converges. The classical stochastic optimization results are included in this theorem. In the second part, the above smoothness assumption on the mean vector field of the algorithm is relaxed using a new approach based on a path-dependent Lyapounov functional. Several applications, for non-smooth mean vector fields and/or bounded Lyapounov function settings, are derived. Examples and simulations are provided that illustrate and enlighten the field of application of the theoretical results.