IMPRECISE MARKOV CHAINS AND THEIR LIMIT BEHAVIOR
- 1 April 2009
- journal article
- research article
- Published by Cambridge University Press (CUP) in Probability in the Engineering and Informational Sciences
- Vol. 23 (4), 597-635
- https://doi.org/10.1017/s0269964809990039
Abstract
When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-calledcredal setsthat these probabilities are known or believed to belong to and by allowing the probabilities to vary over such sets. This leads to the definition of animprecise Markov chain. We show that the time evolution of such a system can be studied very efficiently using so-calledlowerandupper expectations, which are equivalent mathematical representations of credal sets. We also study how the inferred credal set about the state at timenevolves asn→∞: under quite unrestrictive conditions, it converges to a uniquely invariant credal set, regardless of the credal set given for the initial state. This leads to a non-trivial generalization of the classical Perron–Frobenius theorem to imprecise Markov chains.Keywords
Other Versions
This publication has 27 references indexed in Scilit:
- Imprecise probability trees: Bridging two theories of imprecise probabilityArtificial Intelligence, 2008
- Partially observable Markov decision processes with imprecise parametersArtificial Intelligence, 2007
- Graphical models for imprecise probabilitiesInternational Journal of Approximate Reasoning, 2005
- Credal networksArtificial Intelligence, 2000
- How good are convex hull algorithms?Computational Geometry, 1997
- On a Markov chain approach for the study of reliability structuresJournal of Applied Probability, 1996
- Measures of uncertainty in expert systemsArtificial Intelligence, 1996
- On the theory of Markov set-chainsAdvances in Applied Probability, 1994
- Sequential limits in Markov set-chainsJournal of Applied Probability, 1991
- A nonlinear Perron-Frobenius theoremProceedings of the American Mathematical Society, 1990