Universal distribution of saliencies for pruning in layered neural networks.

J. Gorodkin*, L. K. Hansen, B. Lautrup, S. A. Solla

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

6 Scopus citations

Abstract

A better understanding of pruning methods based on a ranking of weights according to their saliency in a trained network requires further information on the statistical properties of such saliencies. We focus on two-layer networks with either a linear or nonlinear output unit, and obtain analytic expressions for the distribution of saliencies and their logarithms. Our results reveal unexpected universal properties of the log-saliency distribution and suggest a novel algorithm for saliency-based weight ranking that avoids the numerical cost of second derivative evaluations.

Original languageEnglish (US)
Pages (from-to)489-498
Number of pages10
JournalInternational journal of neural systems
Volume8
Issue number5-6
DOIs
StatePublished - 1997

ASJC Scopus subject areas

  • Computer Networks and Communications

Fingerprint Dive into the research topics of 'Universal distribution of saliencies for pruning in layered neural networks.'. Together they form a unique fingerprint.

Cite this