Super-resolution of compressed videos using convolutional neural networks

Armin Kappeler, Seunghwan Yoo, Qiqin Dai, Aggelos K. Katsaggelos

Research output: Chapter in Book/Report/Conference proceedingConference contribution

26 Scopus citations

Abstract

Convolutional neural networks (CNN) have been successfully applied to image super-resolution (SR) as well as other image restoration tasks. In this paper, we consider the problem of compressed video super-resolution. Traditional SR algorithms for compressed videos rely on information from the encoder such as frame type or quantizer step, whereas our algorithm only requires the compressed low resolution frames to reconstruct the high resolution video. We propose a CNN that is trained on both the spatial and the temporal dimensions of compressed videos to enhance their spatial resolution. Consecutive frames are motion compensated and used as input to a CNN that provides super-resolved video frames as output. Our network is pretrained with images, which significantly improves the performance over random initialization. In extensive experimental evaluations, we trained the state-of-the-art image and video superresolution algorithms on compressed videos and compared their performance to our proposed method.

Original languageEnglish (US)
Title of host publication2016 IEEE International Conference on Image Processing, ICIP 2016 - Proceedings
PublisherIEEE Computer Society
Pages1150-1154
Number of pages5
ISBN (Electronic)9781467399616
DOIs
StatePublished - Aug 3 2016
Event23rd IEEE International Conference on Image Processing, ICIP 2016 - Phoenix, United States
Duration: Sep 25 2016Sep 28 2016

Publication series

NameProceedings - International Conference on Image Processing, ICIP
Volume2016-August
ISSN (Print)1522-4880

Other

Other23rd IEEE International Conference on Image Processing, ICIP 2016
Country/TerritoryUnited States
CityPhoenix
Period9/25/169/28/16

Keywords

  • Convolutional Neural Networks
  • Deep Learning
  • Super-Resolution
  • Video Compression

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Signal Processing

Fingerprint

Dive into the research topics of 'Super-resolution of compressed videos using convolutional neural networks'. Together they form a unique fingerprint.

Cite this