Regularization for unsupervised deep neural nets

Baiyang Wang, Diego Klabjan

Research output: Contribution to conferencePaperpeer-review

7 Scopus citations

Abstract

Unsupervised neural networks, such as restricted Boltzmann machines (RBMs) and deep belief networks (DBNs), are powerful tools for feature selection and pattern recognition tasks. We demonstrate that overfitting occurs in such models just as in deep feedforward neural networks, and discuss possible regularization methods to reduce overfitting. We also propose a "partial" approach to improve the efficiency of Dropout/DropConnect in this scenario, and discuss the theoretical justification of these methods from model convergence and likelihood bounds. Finally, we compare the performance of these methods based on their likelihood and classification error rates for various pattern recognition data sets.

Original languageEnglish (US)
Pages2681-2687
Number of pages7
StatePublished - Jan 1 2017
Event31st AAAI Conference on Artificial Intelligence, AAAI 2017 - San Francisco, United States
Duration: Feb 4 2017Feb 10 2017

Other

Other31st AAAI Conference on Artificial Intelligence, AAAI 2017
CountryUnited States
CitySan Francisco
Period2/4/172/10/17

ASJC Scopus subject areas

  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Regularization for unsupervised deep neural nets'. Together they form a unique fingerprint.

Cite this