Learning spatio-temporal dependency of local patches for complex motion segmentation

Jiang Xu*, Junsong Yuan, Ying Wu

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

4 Scopus citations


Segmenting complex motion, such as articulated motion and deformable objects, can be difficult if the prior knowledge of the motion pattern is not available. We present a novel method for motion segmentation by learning the motion priors from exemplar motions to guide the segmentation. Instead of modeling the motion field explicitly, we decompose each video frame into a number of local patches and learn the spatio-temporal contextual relations among them, e.g., if their motion relationships are consistent with that from the training data. Based on a novel motion feature to measure the relative motion of two patches, the SVM classifier learns their pairwise relationship. We convert the motion segmentation problem to a binary labeling problem, and propose an iterative solution to group the local patches whose motions are consistent. Compared with other approaches, such as the graph cut and normalized cut methods, this new method is computationally more efficient and is able to better handle the inaccurate inference of pairwise relationships. Results on both synthesized and real videos show that our method can learn to segment different types of complex motion patterns.

Original languageEnglish (US)
Pages (from-to)334-351
Number of pages18
JournalComputer Vision and Image Understanding
Issue number3
StatePublished - Mar 2011


  • Bipolar segmentation
  • Learning
  • Motion profile symmetry correlation
  • Motion segmentation

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition


Dive into the research topics of 'Learning spatio-temporal dependency of local patches for complex motion segmentation'. Together they form a unique fingerprint.

Cite this