Human tracking using convolutional neural networks

Jialue Fan*, Wei Xu, Ying Wu, Yihong Gong

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

300 Scopus citations


In this paper, we treat tracking as a learning problem of estimating the location and the scale of an object given its previous location, scale, as well as current and previous image frames. Given a set of examples, we train convolutional neural networks (CNNs) to perform the above estimation task. Different from other learning methods, the CNNs learn both spatial and temporal features jointly from image pairs of two adjacent frames. We introduce multiple path ways in CNN to better fuse local and global information. A creative shift-variant CNN architecture is designed so as to alleviate the drift problem when the distracting objects are similar to the target in cluttered environment. Furthermore, we employ CNNs to estimate the scale through the accurate localization of some key points. These techniques are object-independent so that the proposed method can be applied to track other types of object. The capability of the tracker of handling complex situations is demonstrated in many testing sequences.

Original languageEnglish (US)
Article number5559504
Pages (from-to)1610-1623
Number of pages14
JournalIEEE Transactions on Neural Networks
Issue number10
StatePublished - Oct 2010


  • Convolutional neural networks
  • machine learning
  • visual tracking

ASJC Scopus subject areas

  • Software
  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications


Dive into the research topics of 'Human tracking using convolutional neural networks'. Together they form a unique fingerprint.

Cite this