Noise injection for training artificial neural networks: A comparison with weight decay and early stopping

Richard M. Zur, Yulei Jiang, Lorenzo L. Pesce, Karen Drukker

Research output: Contribution to journalArticlepeer-review

112 Scopus citations


The purpose of this study was to investigate the effect of a noise injection method on the "overfitting" problem of artificial neural networks (ANNs) in two-class classification tasks. The authors compared ANNs trained with noise injection to ANNs trained with two other methods for avoiding overfitting: weight decay and early stopping. They also evaluated an automatic algorithm for selecting the magnitude of the noise injection. They performed simulation studies of an exclusive-or classification task with training datasets of 50, 100, and 200 cases (half normal and half abnormal) and an independent testing dataset of 2000 cases. They also compared the methods using a breast ultrasound dataset of 1126 cases. For simulated training datasets of 50 cases, the area under the receiver operating characteristic curve (AUC) was greater (by 0.03) when training with noise injection than when training without any regularization, and the improvement was greater than those from weight decay and early stopping (both of 0.02). For training datasets of 100 cases, noise injection and weight decay yielded similar increases in the AUC (0.02), whereas early stopping produced a smaller increase (0.01). For training datasets of 200 cases, the increases in the AUC were negligibly small for all methods (0.005). For the ultrasound dataset, noise injection had a greater average AUC than ANNs trained without regularization and a slightly greater average AUC than ANNs trained with weight decay. These results indicate that training ANNs with noise injection can reduce overfitting to a greater degree than early stopping and to a similar degree as weight decay.

Original languageEnglish (US)
Pages (from-to)4810-4818
Number of pages9
JournalMedical Physics
Issue number10
StatePublished - 2009


  • Artificial neural networks
  • BANN
  • Early stopping
  • Jitter
  • Noise injection
  • Overfitting
  • Regularization
  • Weight decay

ASJC Scopus subject areas

  • Biophysics
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Noise injection for training artificial neural networks: A comparison with weight decay and early stopping'. Together they form a unique fingerprint.

Cite this