A co-inference approach to robust visual tracking

Visual tracking could be treated as a parameter estimation problem of target representation based on observations in image sequences. A richer target representation would incur better chances of successful tracking in cluttered and dynamic environments. However, the dimensionality of target's state space also increases making tracking a formidable estimation problem. In this paper, the problem of tracking and integrating multiple cues is formulated in a probabilistic framework and represented by a factorized graphical model. Structured variational analysis of such graphical model factorizes different modalities and suggests a co-inference process among these modalities. A sequential Monte Carlo algorithm is proposed to give an efficient approximation of the co-inference based on the importance sampling technique. This algorithm is implemented in real-time at around 30Hz. Specifically, tracking both position, shape and color distribution of a target is investigated in this paper. Our extensive experiments show that the proposed algorithm performs robustly in a large variety of tracking scenarios. The approach presented in this paper has the potential to solve other sensor fusion problems.