A consistent algorithm for derivative estimation of Markov chains
- 17 December 2002
- conference paper
- conference paper
- Published by Institute of Electrical and Electronics Engineers (IEEE)
- Vol. 2, 1990-1995 vol.2
- https://doi.org/10.1109/cdc.1994.411084
Abstract
A consistent algorithm for derivative estimation of finite-state, discrete-time Markov chains is presented. The basic idea is to simulate original Markov chains with modified performance measures that can be estimated by extra simulation. The computational load of the extra simulation at each step is bounded. The algorithm attains the best possible rate of convergence as the simulation time goes to infinity. A connection between the algorithm and solutions to Poisson equations is also revealed.Keywords
This publication has 6 references indexed in Scilit:
- Smoothed perturbation analysis derivative estimation for Markov chainsOperations Research Letters, 1994
- Estimating Derivatives Via Poisson's EquationProbability in the Engineering and Informational Sciences, 1991
- Perturbation Analysis of Discrete Event Dynamic SystemsPublished by Springer Science and Business Media LLC ,1991
- Discrete-time ‘inversion’ and derivative estimation for Markov chainsOperations Research Letters, 1990
- On-line sensitivity analysis of Markov chainsIEEE Transactions on Automatic Control, 1989
- Maximal couplingProbability Theory and Related Fields, 1979