GRAIL: Generative Range and Altitude Identity Learning

Objective: Incorporate AI approaches to enhance current diagnostic tools/methods and build a device for monitoring respiratory activities Summary: We propose to develop an innovative deep neural network architecture to simultaneously address the removal of the motion blur (motion deblurring -MDB), the atmospheric turbulence mitigation (ATM), and the video super-resolution (VSR) problems, henceforth referred to as the VSR/ATM/MDB enhancement system. Such a system will address the enhancement of the whole scene or the detected faces. It will be trained separately from the remaining components of the BRIAR system, henceforth referred to as the baseline model, and simultaneously with the remaining components of the BRIAR system –end-to-end system--. The reason for considering both configurations is that during the initial phases the baseline VSR/ATM/MBD enhancement system will be brought to a level of maturity by one group while other groups will be developing the remaining components of the BRIAR system. After both components reach a level of maturity the more complicated end-to-end system will be trained simultaneously. This second step will be considerably shorter since the individual components will have been already developed and individually trained. The baseline and end-to-end systems will be analyzed and thoroughly compared experimentally. Approach: We propose to solve the inverse problem described above by extending our previous results. We believe that these architectures we have already developed present critical components for solving the problem at hand. More precisely, the GAN architecture we have developed can be used as a starting point for developing the new architecture. The combination of the residual architecture and perceptual based losses allows this model to perform super-resolution for high scaling factors while still producing high perceptual quality video sequences. To accommodate the geometric deformation caused by the atmospheric turbulence, we propose to use the extension of a model we introduced along with recent approaches which appeared in the literature. However, these architectures were developed for less challenging scenarios, in which the degradation, down-sampling, and noise were less severe. To produce video sequences appropriate for the BRIAR system, the model needs to exploit information about the subject contained in a long duration video sequence. The majority of the current video enhancement networks are only able to exploit short-term information. We therefore propose to expand the attention mechanism we introduced by using the transformer architecture. Timeframe: 8/1/21-7/31/25 Deliverable(s): The deliverables will be software and reports. Reporting: The progress of the work will be reported at face-to-face meetings, and plenary teleconferences will be scheduled on a periodic basis.