DeepBinaryMask: Learning a binary mask for video compressive sensing

Michael Iliadis*, Leonidas Spinoulas, Aggelos K. Katsaggelos

*Corresponding author for this work

Research output: Contribution to journalArticle

1 Scopus citations

Abstract

In this paper, we propose an encoder-decoder neural network model referred to as DeepBinaryMask for video compressive sensing. In video compressive sensing one frame is acquired using a set of coded masks (sensing matrix) from which a number of video frames, equal to the number of coded masks, is reconstructed. The proposed framework is an end-to-end model where the sensing matrix is trained along with the video reconstruction. The encoder maps a video block to compressive measurements by learning the binary elements of the sensing matrix. The decoder is trained to map the measurements from a video patch back to a video block via several hidden layers of a Multi-Layer Perceptron network. The predicted video blocks are stacked together to recover the unknown video sequence. The reconstruction performance is found to improve when using the trained sensing mask from the network as compared to other mask designs such as random, across a wide variety of compressive sensing reconstruction algorithms. Finally, our analysis and discussion offers insights into understanding the characteristics of the trained mask designs that lead to the improved reconstruction quality.

Original languageEnglish (US)
Article number102591
JournalDigital Signal Processing: A Review Journal
Volume96
DOIs
StatePublished - Jan 2020

    Fingerprint

Keywords

  • Binary mask
  • Compressive sensing
  • Deep learning
  • Mask optimization
  • Video reconstruction

ASJC Scopus subject areas

  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Statistics, Probability and Uncertainty
  • Computational Theory and Mathematics
  • Electrical and Electronic Engineering
  • Artificial Intelligence
  • Applied Mathematics

Cite this