The Maclaurin series for performance functions of Markov chains
- 1 September 1998
- journal article
- Published by Cambridge University Press (CUP) in Advances in Applied Probability
- Vol. 30 (3), 676-692
- https://doi.org/10.1239/aap/1035228123
Abstract
We derive formulas for the first- and higher-order derivatives of the steady state performance measures for changes in transition matrices of irreducible and aperiodic Markov chains. Using these formulas, we obtain a Maclaurin series for the performance measures of such Markov chains. The convergence range of the Maclaurin series can be determined. We show that the derivatives and the coefficients of the Maclaurin series can be easily estimated by analysing a single sample path of the Markov chain. Algorithms for estimating these quantities are provided. Markov chains consisting of transient states and multiple chains are also studied. The results can be easily extended to Markov processes. The derivation of the results is closely related to some fundamental concepts, such as group inverse, potentials, and realization factors in perturbation analysis. Simulation results are provided to illustrate the accuracy of the single sample path based estimation. Possible applications to engineering problems are discussed.Keywords
This publication has 11 references indexed in Scilit:
- Perturbation realization, potentials, and sensitivity analysis of Markov processesIEEE Transactions on Automatic Control, 1997
- A formula for singular perturbations of Markov chainsJournal of Applied Probability, 1994
- Nonnegative Matrices in the Mathematical SciencesPublished by Society for Industrial & Applied Mathematics (SIAM) ,1994
- A new approach to (s, S) inventory systemsJournal of Applied Probability, 1993
- The MacLaurin series for the GI/G/1 queueJournal of Applied Probability, 1992
- Exact formula for sensitivity analysis of Markov chainsJournal of Optimization Theory and Applications, 1991
- Perturbation Analysis of Discrete Event Dynamic SystemsPublished by Springer Science and Business Media LLC ,1991
- A numerical approach to cyclic-service queueing modelsQueueing Systems, 1990
- Power Series for Stationary Distributions of Coupled Processor ModelsSIAM Journal on Applied Mathematics, 1988
- The Role of the Group Generalized Inverse in the Theory of Finite Markov ChainsSIAM Review, 1975