Estimation of the information by an adaptive partitioning of the observation space
- 1 May 1999
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Information Theory
- Vol. 45 (4), 1315-1321
- https://doi.org/10.1109/18.761290
Abstract
We demonstrate that it is possible to approximate the mutual information arbitrarily closely in probability by calculating the relative frequencies on appropriate partitions and achieving conditional independence on the rectangles of which the partitions are made. Empirical results, including a comparison with maximum-likelihood estimators, are presented.Keywords
This publication has 7 references indexed in Scilit:
- The mutual information as a measure of statistical dependencePublished by Institute of Electrical and Electronics Engineers (IEEE) ,2002
- Elements of Information TheoryPublished by Wiley ,2001
- Predictability: An Information-Theoretic PerspectivePublished by Springer Science and Business Media LLC ,1998
- Limit theorems for the logarithm of sample spacingsStatistics & Probability Letters, 1995
- Multivariate Density EstimationWiley Series in Probability and Statistics, 1992
- Distribution estimation consistent in total variation and in two types of information divergenceIEEE Transactions on Information Theory, 1992
- Density-free convergence properties of various estimators of entropyComputational Statistics & Data Analysis, 1987