Accelerating Nonconvex Learning via Replica Exchange Langevin Diffusion

Yi Chen, Jinglin Chen, Jing Dong, Jian Peng, Zhaoran Wang

Research output: Contribution to journalArticlepeer-review

Abstract

Langevin diffusion is a powerful tool for nonconvex optimization problems, which can be used to find the global minima. However, the standard Langevin diffusion driven by a single temperature suffers from the tradeoff between “global exploration” and “local exploitation”, corresponding the high and low temperatures, respectively. In order to bridge such a gap, we propose to use the replica exchange Langevin diffusion for the purpose of nonconvex optimization, where two Langevin diffusions run simultaneously with positions swapping. We show that, compared with the standard Langevin diffusion, replica exchange enables us to approach the global minima faster through accelerating the convergence of Langevin diffusion. We also propose a novel optimization algorithm by discretizing the replica exchange Langevin diffusion.

Original languageEnglish (US)
JournalUnknown Journal
StatePublished - Jul 3 2020

ASJC Scopus subject areas

  • General

Fingerprint Dive into the research topics of 'Accelerating Nonconvex Learning via Replica Exchange Langevin Diffusion'. Together they form a unique fingerprint.

Cite this