Abstract
Consider arbitrarily distributed input signals observed in additive Gaussian noise. A new fundamental relationship is found between the input-output mutual information and the minimum mean-square error (MMSE) of an estimate of the input given the output: The derivative of the mutual information (nats) with respect to the signal-to-noise ratio (SNR) is equal to half the MMSE. This identity holds for both scalar and vector signals, as well as for discrete- and continuous-time noncausal MMSE estimation (smoothing). A consequence of the result is a new relationship in continuous-time nonlinear filtering: Regardless of the input statistics, the causal MMSE achieved at snr is equal to the expected value of the noncausal MMSE achieved with a channel whose SNR is chosen uniformly distributed between 0 and snr.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 347 |
| Number of pages | 1 |
| Journal | IEEE International Symposium on Information Theory - Proceedings |
| State | Published - 2004 |
| Event | Proceedings - 2004 IEEE International Symposium on Information Theory - Chicago, IL, United States Duration: Jun 27 2004 → Jul 2 2004 |
ASJC Scopus subject areas
- Theoretical Computer Science
- Information Systems
- Modeling and Simulation
- Applied Mathematics