Robust visual tracking by integrating multiple cues based on co-inference learning

Ying Wu*, Thomas S. Huang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

125 Scopus citations


Visual tracking can be treated as a parameter estimation problem that infers target states based on image observations from video sequences. A richer target representation may incur better chances of successful tracking in cluttered and dynamic environments, and thus enhance the robustness. Richer representations can be constructed by either specifying a detailed model of a single cue or combining a set of rough models of multiple cues. Both approaches increase the dimensionality of the state space, which results in a dramatic increase of computation. To investigate the integration of rough models from multiple cues and to explore computationally efficient algorithms, this paper formulates the problem of multiple cue integration and tracking in a probabilistic framework based on a factorized graphical model. Structured variational analysis of such a graphical model factorizes different modalities and suggests a co-inference process among these modalities. Based on the importance sampling technique, a sequential Monte Carlo algorithm is proposed to provide an efficient simulation and approximation of the co-inferencing of multiple cues. This algorithm runs in real-time at around 30 Hz. Our extensive experiments show that the proposed algorithm performs robustly in a large variety of tracking scenarios. The approach presented in this paper has the potential to solve other problems including sensor fusion problems.

Original languageEnglish (US)
Pages (from-to)55-71
Number of pages17
JournalInternational Journal of Computer Vision
Issue number1
StatePublished - Jun 2004


  • Co-inference
  • Factorized graphical model
  • Importance sampling
  • Sequential Monte Carlo
  • Variational analysis
  • Visual tracking

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Robust visual tracking by integrating multiple cues based on co-inference learning'. Together they form a unique fingerprint.

Cite this