Rate-distortion optimal video summary generation

Zhu Li*, Guido M. Schuster, Aggelos K. Katsaggelos, Bhavan Gandhi

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

68 Scopus citations


The need for video summarization originates primarily from a viewing time constraint. A shorter version of the original video sequence is desirable in a number of applications. Clearly, a shorter version is also necessary in applications where storage, communication bandwidth, and/or power are limited. The summarization process inevitably introduces distortion. The amount of summarization distortion is related to its "conciseness," or the number of frames available in the summary. If there are m frames in the original sequence and n frames in the summary, we define the summarization rate as m/n, to characterize this "conciseness." We also develop a new summarization distortion metric and formulate the summarization problem as a rate-distortion optimization problem. Optimal algorithms based on dynamic programming are presented and compared experimentally with heuristic algorithms. Practical constraints, like the maximum number of frames that can be skipped, are also considered in the formulation and solution of the problem.

Original languageEnglish (US)
Pages (from-to)1550-1560
Number of pages11
JournalIEEE Transactions on Image Processing
Issue number10
StatePublished - Oct 1 2005


  • Dynamic programming (DP)
  • Rate-distortion optimization
  • Video analysis
  • Video summarization

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Rate-distortion optimal video summary generation'. Together they form a unique fingerprint.

Cite this