Perceptually based techniques for semantic image classification and retrieval

Dejan Depalov*, Thrasyvoulos Pappas, Dongge Li, Bhavan Gandhi

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

19 Scopus citations


The accumulation of large collections of digital images has created the need for efficient and intelligent schemes for content-based image retrieval. Our goal is to organize the contents semantically, according to meaningful categories. We present a new approach for semantic classification that utilizes a recently proposed color-texture segmentation algorithm (by Chen et al.), which combines knowledge of human perception and signal characteristics to segment natural scenes into perceptually uniform regions. The color and texture features of these regions are used as medium level descriptors, based on which we extract semantic labels, first at the segment and then at the scene level. The segment features consist of spatial texture orientation information and color composition in terms of a limited number of locally adapted dominant colors. The focus of this paper is on region classification. We use a hierarchical vocabulary of segment labels that is consistent with those used in the NIST TRECVID 2003 development set. We test the approach on a database of 9000 segments obtained from 2500 photographs of natural scenes. For training and classification we use the Linear Discriminant Analysis (LDA) technique. We examine the performance of the algorithm (precision and recall rates) when different sets of features (e.g., one or two most dominant colors versus four quantized dominant colors) are used. Our results indicate that the proposed approach offers significant performance improvements over existing approaches.

Original languageEnglish (US)
Title of host publicationHuman Vision and Electronic Imaging XI - Proceedings of SPIE-IS and T Electronic Imaging
StatePublished - 2006
EventHuman Vision and Electronic Imaging XI - San Jose, CA, United States
Duration: Jan 16 2006Jan 18 2006

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
ISSN (Print)0277-786X


OtherHuman Vision and Electronic Imaging XI
Country/TerritoryUnited States
CitySan Jose, CA


  • Adaptive clustering
  • Image analysis
  • Perceptual models
  • Retrieval
  • Segmentation
  • Semantic classification
  • Steerable filters

ASJC Scopus subject areas

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering


Dive into the research topics of 'Perceptually based techniques for semantic image classification and retrieval'. Together they form a unique fingerprint.

Cite this