Semi-tied covariance matrices for hidden Markov models
- 1 May 1999
- journal article
- Published by Institute of Electrical and Electronics Engineers (IEEE) in IEEE Transactions on Speech and Audio Processing
- Vol. 7 (3), 272-281
- https://doi.org/10.1109/89.759034
Abstract
There is normally a simple choice made in the form of the covariance matrix to be used with continuous-density HMMs. Either a diagonal covariance matrix is used, with the underlying assumption that elements of the feature vector are independent, or a full or block-diagonal matrix is used, where all or some of the correlations are explicitly modeled. Unfortunately when using full or block-diagonal covariance matrices there tends to be a dramatic increase in the number of parameters per Gaussian component, limiting the number of components which may be robustly estimated. This paper introduces a new form of covariance matrix which allows a few "full" covariance matrices to be shared over many distributions, whilst each distribution maintains its own "diagonal" covariance matrix. In contrast to other schemes which have hypothesized a similar form, this technique fits within the standard maximum-likelihood criterion used for training HMMs. The new form of covariance matrix is evaluated on a large-vocabulary speech-recognition task. In initial experiments the performance of the standard system was achieved using approximately half the number of parameters. Moreover, a 10% reduction in word error rate compared to a standard system can be achieved with less than a 1% increase in the number of parameters and little increase in recognition time.Keywords
This publication has 10 references indexed in Scilit:
- Mean and variance adaptation within the MLLR frameworkComputer Speech & Language, 1996
- Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov modelsComputer Speech & Language, 1995
- The importance of cepstral parameter correlations in speech recognitionComputer Speech & Language, 1994
- A one pass decoder design for large vocabulary recognitionPublished by Association for Computational Linguistics (ACL) ,1994
- Tree-based state tying for high accuracy acoustic modellingPublished by Association for Computational Linguistics (ACL) ,1994
- Context dependent modeling of phones in continuous speech using decision treesPublished by Association for Computational Linguistics (ACL) ,1991
- A tutorial on hidden Markov models and selected applications in speech recognitionProceedings of the IEEE, 1989
- Maximum-Likelihood Estimation for Mixture Multivariate Stochastic Observations of Markov ChainsAT&T Technical Journal, 1985
- Maximum likelihood estimation for multivariate observations of Markov sourcesIEEE Transactions on Information Theory, 1982
- Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentencesIEEE Transactions on Acoustics, Speech, and Signal Processing, 1980