Stochastic Constrained Optimization

Machine learning, signal processing, and control are among the many applications that require the solution of optimization problems under uncertainty. Most of the stochastic optimization models studied in these disciplines, particularly in machine learning, are unconstrained, but there is increasing interest in including domain-specific constraints that in turn may also contain uncertainty. The goal of this proposal is to develop new optimization methods for applications of this kind, i.e., for stochastic, constrained optimization problems. This is a relatively unexplored area of nonlinear optimization that poses significant challenges, especially in the large-scale setting. The main thrust of this proposal consists of developing algorithms that are well grounded in theory and are also efficient and robust over a range of applications. The project encompasses two other topics that are closely related to the main purpose of the investigation. The first concerns the case when the objective is given by a black box that does not provide derivatives, but only provides noisy function evaluations. Engineering problems of this kind abound; one example is the minimization of a reward function in reinforcement learning. The second topic focuses on the design of new quasi-Newton methods that can tolerate noise in the function and gradient evaluation. Classical techniques break down in this case because noisy curvature information can yield unreliable Hessian updates that render the optimization iteration ineffective. The goal is to design noise-tolerant quasi-Newton methods that reduce to the classical counterparts in the absence of noise but are robust and efficient when noise is present. The effectiveness of the proposed algorithms will be demonstrated on realistic applications arising in machine learning, control and signal processing.