Estimating the Infinitesimal Generator of a Continuous Time, Finite State Markov Process

Abstract

Let ${Z(t), t > 0}$ be a separable, continuous time Markov Process with stationary transition probabilities $P_{ij}(t), i, j = 1, 2, cdots, M$. Under suitable regularity conditions, the matrix of transition probabilities, $P(t)$, can be expressed in the form $P(t) = exp tQ$, where $Q$ is an $M imes M$ matrix and is called the "infinitesimal generator" for the process. In this paper, a density on the space of sample functions over $[0, t)$ is constructed. This density depends upon $Q$. If $Q$ is unknown, the maximum likelihood estimate $hat{Q}(k, t) = |hat{q}_{ij}(k, t)|$, based upon $k$ independent realizations of the process over $lbrack 0, t)$ can be derived. If each state has positive probability of being occupied during $lbrack 0, t)$ and if the number of independent observations, $k$, grows larger ($t$ held fixed), then $hat{q}_{ij}$ is strongly consistent and the joint distribution of the set ${k^{frac{1}{2}}(hat{q}_{ij} - q_{ij})}_{i eq j}$ (suitably normalized), is asymptotically normal with zero mean and covariance equal to the identity matrix. If $k$ is held fixed (at one, say) and if $t$ grows large, then $hat{q}_{ij}$ is again strongly consistent and the joint distribution of the set ${t^{frac{1}{2}}(hat{q}_{ij} - q_{ij})}_{i eq j}$ (suitably normalized), is asymptotically normal with zero mean and covariance equal to the identity matrix, provided that the process ${Z(t), t > 0}$ is positively regular. The asymptotic variances of the $hat{q}_{ij}$ are computed in both cases.