Nonlinear principal component analysis by neural networks

Abstract

Nonlinear principal component analysis (NLPCA) can be performed by a neural network modelwhich nonlinearly generalizes the classical principal component analysis (PCA) method. Thepresence of local minima in the cost function renders the NLPCA somewhat unstable, asoptimizations started from different initial parameters often converge to different minima.Regularization by adding weight penalty terms to the cost function is shown to improve thestability of the NLPCA. With the linear approach, there is a dichotomy between PCA androtated PCA methods, as it is generally impossible to have a solution simultaneously(a) explaining maximum global variance of the data, and (b) approaching local data clusters.With the NLPCA, both objectives (a) and (b) can be attained together, thus the nonlinearityin NLPCA unifies the PCA and rotated PCA approaches. With a circular node at the networkbottleneck, the NLPCA is able to extract periodic or wave modes. The Lorenz (1963)3-component chaotic system and the monthly tropical Pacific sea surface temperatures(1950-1999) are used to illustrated the NLPCA approach. DOI: 10.1034/j.1600-0870.2001.00251.x