Unsupervised Hierarchical Dynamic Parsing and Encoding for Action Recognition

Bing Su*, Jiahuan Zhou, Xiaoqing Ding, Ying Wu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

17 Scopus citations


Generally, the evolution of an action is not uniform across the video, but exhibits quite complex rhythms and non-stationary dynamics. To model such non-uniform temporal dynamics, in this paper, we describe a novel hierarchical dynamic parsing and encoding method to capture both the locally smooth dynamics and globally drastic dynamic changes. It parses the dynamics of an action into different layers and encodes such multi-layer temporal information into a joint representation for action recognition. At the first layer, the action sequence is parsed in an unsupervised manner into several smooth-changing stages corresponding to different key poses or temporal structures by temporal clustering. The dynamics within each stage are encoded by mean-pooling or rank-pooling. At the second layer, the temporal information of the ordered dynamics extracted from the previous layer is encoded again by rank-pooling to form the overall representation. Extensive experiments on a gesture action data set (Chalearn Gesture) and three generic action data sets (Olympic Sports, Hollywood2, and UCF101) have demonstrated the effectiveness of the proposed method.

Original languageEnglish (US)
Article number8016651
Pages (from-to)5784-5799
Number of pages16
JournalIEEE Transactions on Image Processing
Issue number12
StatePublished - Dec 2017


  • Action recognition
  • dynamic encoding
  • hierarchical modeling
  • temporal clustering

ASJC Scopus subject areas

  • Software
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Unsupervised Hierarchical Dynamic Parsing and Encoding for Action Recognition'. Together they form a unique fingerprint.

Cite this