Limitations of nonlinear PCA as performed with generic neural networks

Edward C. Malthouse*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

97 Scopus citations

Abstract

Kramer's nonlinear principal components analysis (NLPCA) neural networks are feedforward autoassociative networks with five layers. The third layer has fewer nodes than the input or output layers. This paper proposes a geometric interpretation for Kramer's method by showing that NLPCA fits a lower-dimensional curve or surface through the training data. The first three layers project observations onto the curve or surface giving scores. The last three layers define the curve or surface. The first three layers are a continuous function, which we show has several implications: NLPCA "projections" are suboptimal producing larger approximation error, NLPCA is unable to model curves and surfaces that intersect themselves, and NLPCA cannot parameterize curves with parameterizations having discontinuous jumps. We establish results on the identification of score values and discuss their implications on interpreting score values. We discuss the relationship between NLPCA and principal curves and surfaces, another nonlinear feature extraction method.

Original languageEnglish (US)
Pages (from-to)165-173
Number of pages9
JournalIEEE Transactions on Neural Networks
Volume9
Issue number1
DOIs
StatePublished - 1998

Keywords

  • Data compression
  • Feature extraction
  • Nonlinear principal components analysis
  • Principal components
  • Principal curves
  • Principal surfaces

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Limitations of nonlinear PCA as performed with generic neural networks'. Together they form a unique fingerprint.

Cite this