Universal coding, information, prediction, and estimation
- 1 July 1984
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 30 (4), 629-636
- https://doi.org/10.1109/tit.1984.1056936
Abstract
A connection between universal codes and the problems of prediction and statistical estimation is established. A known lower bound for the mean length of universal codes is sharpened and generalized, and optimum universal codes constructed. The bound is defined to give the information in strings relative to the considered class of processes. The earlier derived minimum description length criterion for estimation of parameters, including their number, is given a fundamental information, theoretic justification by showing that its estimators achieve the information in the strings. It is also shown that one cannot do prediction in Gaussian autoregressive moving average (ARMA) processes below a bound, which is determined by the information in the data.Keywords
This publication has 15 references indexed in Scilit:
- A universal data compression systemIEEE Transactions on Information Theory, 1983
- A Universal Prior for Integers and Estimation by Minimum Description LengthThe Annals of Statistics, 1983
- Minimax noiseless universal coding for Markov sourcesIEEE Transactions on Information Theory, 1983
- Efficient universal noiseless source codesIEEE Transactions on Information Theory, 1981
- Universal modeling and codingIEEE Transactions on Information Theory, 1981
- Asymptotically Efficient Selection of the Order of the Model for Estimating Parameters of a Linear ProcessThe Annals of Statistics, 1980
- Modeling by shortest data descriptionAutomatica, 1978
- Some equivalences between Shannon entropy and Kolmogorov complexityIEEE Transactions on Information Theory, 1978
- Universal noiseless codingIEEE Transactions on Information Theory, 1973
- The prediction error of stationary Gaussian time series of unknown covarianceIEEE Transactions on Information Theory, 1965