Discriminative video pattern search for efficient action detection

Junsong Yuan*, Zicheng Liu, Ying Wu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

127 Scopus citations

Abstract

Actions are spatiotemporal patterns. Similar to the sliding window-based object detection, action detection finds the reoccurrences of such spatiotemporal patterns through pattern matching, by handling cluttered and dynamic backgrounds and other types of action variations. We address two critical issues in pattern matching-based action detection: 1) the intrapattern variations in actions, and 2) the computational efficiency in performing action pattern search in cluttered scenes. First, we propose a discriminative pattern matching criterion for action classification, called naive Bayes mutual information maximization (NBMIM). Each action is characterized by a collection of spatiotemporal invariant features and we match it with an action class by measuring the mutual information between them. Based on this matching criterion, action detection is to localize a subvolume in the volumetric video space that has the maximum mutual information toward a specific action class. A novel spatiotemporal branch-and-bound (STBB) search algorithm is designed to efficiently find the optimal solution. Our proposed action detection method does not rely on the results of human detection, tracking, or background subtraction. It can handle action variations such as performing speed and style variations as well as scale changes well. It is also insensitive to dynamic and cluttered backgrounds and even to partial occlusions. The cross-data set experiments on action detection, including KTH, CMU action data sets, and another new MSR action data set, demonstrate the effectiveness and efficiency of the proposed multiclass multiple-instance action detection method.

Original languageEnglish (US)
Article number5719621
Pages (from-to)1000-9999
Number of pages9000
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume33
Issue number9
DOIs
StatePublished - 2011

Keywords

  • Video pattern search
  • action detection
  • spatiotemporal branch-and-bound search

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Computational Theory and Mathematics
  • Artificial Intelligence
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Discriminative video pattern search for efficient action detection'. Together they form a unique fingerprint.

Cite this