Series Expansions for Continuous-Time Markov Processes

Abstract

We present update formulas that allow us to express the stationary distribution of a continuous-time Markov process with denumerable state space having generator matrix Q^* through a continuous-time Markov process with generator matrix Q. Under suitable stability conditions, numerical approximations can be derived from the update formulas, and we show that the algorithms converge at a geometric rate. Applications to sensitivity analysis and bounds on perturbations are discussed as well. Numerical examples are presented to illustrate the efficiency of the proposed algorithm.