On the generalized Karhunen-Loeve expansion (Corresp.)

Abstract

In a recent paper Watanabe (1965) has established two useful theorems concerning the optimum properties of the Karhunen-Loeve expansion for the random functions of a stochastic process. The resulting coordinate system (Karhunen-Loeve system) was found to be optimum in the sense of i) minimizing the mean-square error committed by approximating the expansion of an infinite series by a finite number of terms, ii) minimizing the entropy function defined over the probability distribution of the coordinate coefficients for the entire ensemble. These optimum features were then effectively applied to the preprocessing of input data for the speech recognition problem. However, in the development of these properties considerations were only given to the situation where the random functions come from the same stochastic process, and consequently the coordinate coefficients in the expansion were treated to be nonrandom. The question then arises as to whether the optimum properties will hold if the random functions are realizations of more than one stochastic process as in the case of many preprocessing problems in pattern recognization and signal detection. The purpose of this correspondence is to show. that the cited optimum properties can be retained by defining a generalized Karhunen-Loeve expansion which considers the possibility of two or more stochastic processes generating the random functions. Necessary conditions are derived to assure the existence of such an expansion. Applications of these results are indicated in the ranking and selection of feature measurements for the sequential recognition (decision) problems.