Nonstationary color tracking for vision-based human - Computer interaction

Ying Wu*, Thomas S. Huang

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

53 Scopus citations


Skin color offers a strong cue for efficient localization and tracking of human body parts in video sequences for vision-based human-computer interaction. Color-based target localization could be achieved by analyzing segmented skin color regions. However, one of the challenges of color-based target tracking is that color distributions would change in different lighting conditions such that fixed color models would be inadequate to capture nonstationary color distributions over time. Meanwhile, using a fixed skin color model trained by the data of a specific person would probably not work well for other people. Although some work has been done on adaptive color models, this problem still needs further studies. This paper presents our investigation of color-based image segmentation and nonstationary color-based target tracking, by studying two different representations for color distributions. In this paper, we propose the structure adaptive self-organizing map (SASOM) neural network that serves as a new color model. Our experiments show that such a representation is powerful for efficient image segmentation. Then, we formulate the nonstationary color tracking problem as a model transduction problem, the solution of which offers a way to adapt and transduce color classifiers in nonstationary color distributions. To fulfill model transduction, this paper proposes two algorithms, the SASOM transduction and the discriminant expectation-maximazation (EM), based on the SASOM color model and the Gaussian mixture color model, respectively. Our extensive experiments on the task of real-time face/hand localization show that these two algorithms can successfully handle some difficulties in nonstationary color tracking. We also implemented a real-time face/hand localization system based on such algorithms for vision-based human-computer interaction.

Original languageEnglish (US)
Pages (from-to)948-960
Number of pages13
JournalIEEE Transactions on Neural Networks
Issue number4
StatePublished - Jul 2002


  • Color model
  • Color-based image segmentation
  • Discriminant analysis
  • Expectation-maximization (EM)
  • Nonstationary color tracking
  • Structure adaptive self-organizing map (SASOM)
  • Vision-based human-computer interaction

ASJC Scopus subject areas

  • Software
  • Computer Science Applications
  • Computer Networks and Communications
  • Artificial Intelligence

Fingerprint Dive into the research topics of 'Nonstationary color tracking for vision-based human - Computer interaction'. Together they form a unique fingerprint.

Cite this