Limitations of nonlinear PCA as performed with generic neural networks

Edward C. Malthouse*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

103 Scopus citations

Abstract

Kramer's nonlinear principal components analysis (NLPCA) neural networks are feedforward autoassociative networks with five layers. The third layer has fewer nodes than the input or output layers. This paper proposes a geometric interpretation for Kramer's method by showing that NLPCA fits a lower-dimensional curve or surface through the training data. The first three layers project observations onto the curve or surface giving scores. The last three layers define the curve or surface. The first three layers are a continuous function, which we show has several implications: NLPCA "projections" are suboptimal producing larger approximation error, NLPCA is unable to model curves and surfaces that intersect themselves, and NLPCA cannot parameterize curves with parameterizations having discontinuous jumps. We establish results on the identification of score values and discuss their implications on interpreting score values. We discuss the relationship between NLPCA and principal curves and surfaces, another nonlinear feature extraction method.

Original languageEnglish (US)
Pages (from-to)165-173
Number of pages9
JournalIEEE Transactions on Neural Networks
Volume9
Issue number1
DOIs
StatePublished - 1998

Funding

Manuscript received September 23, 1996; revised April 30, 1997 and September 4, 1997. The computing equipment used for the empirical results presented in this paper was funded in part by NSF Grant DMS-9505799. The author is with Northwestern University, Evanston, IL 60208-2001 USA. Publisher Item Identifier S 1045-9227(98)01049-2.

Keywords

  • Data compression
  • Feature extraction
  • Nonlinear principal components analysis
  • Principal components
  • Principal curves
  • Principal surfaces

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Limitations of nonlinear PCA as performed with generic neural networks'. Together they form a unique fingerprint.

Cite this