RI: Small:Modeling and Learning Visual Similarities under Adverse Visual Conditions

RI: Small: Modeling and Learning Visual Similarities in the Wild Di�erent from visual classi�cation, visual regression pursues the mapping from the high- dimensional visual space to another continuous space for more �ne-grained visual inference. It has wide applications in vision tasks, but it remains to be a very challenging problem. Traditional approaches attempt to use a holistic parameterized function to �t the training data directly, and thus they are painstaking o�-line learning processes. With the rapid growth of computational power, it is very worthwhile to explore a new methodology that shifts part of the learning from o�-line to on-line computation. The core idea is to steer the global regression on-the- y to local regression that is only performed on a small set of exemplars retrieved from a large exemplar database. Studying this "regression-by-search" approach is the main focus of this proposal. Despite the its exibility in learning and adaptation, the study of this new approach is still in its early stage, and many fundamental issues need to be addressed, including visual similarity, regression modeling, and handling massive visual data, etc. The long-term goal of this project is to lay the foundation of modeling and learning space metric for visual regression, by pursuing innovative "regression-by-search" approaches to providing exible and adaptive solutions to di�cult visual regression tasks, to facilitating analyzable and explainable regression, and to empowering many emerging and versatile applications. The proposed research will lead to more e�ective methods for visual regression. Key Words: computer vision; visual regression; image super resolution; gaze estimation Intellectual Merit: The major objective is to integrate modeling and learning for visual regression, and to provide methods and tools for real-life applications. It is focused on: � Modeling the integration of visual regression and search for analyzable and explainable regression. This novel modeling decomposes the complex regression into several analyzable components including metric learning for space alignment, visual similarity, robust visual search, local reconstruction and transferring. � Learning space metric for space alignment for visual similarity and visual regression. It pursues to answer an important question: under what condition and how the local recon- struction can be transferred? The novel research is to explore and model the structural relationship between the input and output spaces, which we call space alignment. It facili- tates to improve the quality of visual search, and to better regularize local reconstruction. � Two solid case studies on challenging tasks for validation. One is single-image super resolution, which is a very di�cult ill-posed visual regression problem. And the other is remote accurate gaze estimation. The proposed research is validated and evaluated on these two case studies. Broader Impact: The project is expected to signi�cantly advance the research of visual regres- sion and visual similarity, and to result in an important enabling technology for a wide range of applications including visual search, intelligent surveillance and security, human-computer interaction, visual communications, etc. In addition, the research of image SR and gaze is also helpful for the visually impaired for better visual sensing abilities. This research program will contribute to education through curriculum development, student involvements, and workshops and tutorials outside the vision