Statistical physics of signal estimation in gaussian noise: Theory and examples of phase transitions

Neri Merhav*, Dongning Guo, Shlomo Shamai

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

36 Scopus citations

Abstract

We consider the problem of signal estimation (denoising) from a statistical mechanical perspective, using a relationship between the minimum mean square error (MMSE), of estimating a signal, and the mutual information between this signal and its noisy version. The paper consists of essentially two parts. In the first, we derive several statistical-mechanical relationships between a few important quantities in this problem area, such as the MMSE, the differential entropy, the Fisher information, the free energy, and a generalized notion of temperature. We also draw analogies and differences between certain relations pertaining to the estimation problem and the parallel relations in thermodynamics and statistical physics. In the second part of the paper, we provide several application examples, where we demonstrate how certain analysis tools that are customary in statistical physics, prove useful in the analysis of the MMSE. In most of these examples, the corresponding statistical-mechanical systems turn out to consist of strong interactions that cause phase transitions, which in turn are reflected as irregularities and discontinuities (similar to threshold effects) in the behavior of the MMSE.

Original languageEnglish (US)
Article number5429116
Pages (from-to)1400-1416
Number of pages17
JournalIEEE Transactions on Information Theory
Volume56
Issue number3
DOIs
StatePublished - Mar 2010

Keywords

  • Denoising, de Bruijns identity
  • Gaussian channel
  • Minimum mean square error (MMSE) estimation
  • Phase transitions
  • Random energy model
  • Spin glasses
  • Statistical mechanics

ASJC Scopus subject areas

  • Information Systems
  • Computer Science Applications
  • Library and Information Sciences

Fingerprint

Dive into the research topics of 'Statistical physics of signal estimation in gaussian noise: Theory and examples of phase transitions'. Together they form a unique fingerprint.

Cite this