Extracting and using attribute-value pairs from product descriptions on the Web

Katharina Probst*, Rayid Ghani, Marko Krema, Andy Fano, Yan Liu

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations


We describe an approach to extract attribute-value pairs from product descriptions in order to augment product databases by representing each product as a set of attribute-value pairs. Such a representation is useful for a variety of tasks where treating a product as a set of attribute-value pairs is more useful than as an atomic entity. We formulate the extraction task as a classification problem and use Naïve Bayes combined with a multi-view semi-supervised algorithm (co-EM). The extraction system requires very little initial user supervision: using unlabeled data, we automatically extract an initial seed list that serves as training data for the semi-supervised classification algorithm. The extracted attributes and values are then linked to form pairs using dependency information and co-location scores. We present promising results on product descriptions in two categories of sporting goods products. The extracted attribute-value pairs can be useful in a variety of applications, including product recommendations, product comparisons, and demand forecasting. In this paper, we describe one practical application of the extracted attribute-value pairs: a prototype of an Assortment Comparison Tool that allows retailers to compare their product assortments to those of their competitors. As the comparison is based on attributes and values, we can draw meaningful conclusions at a very fine-grained level. We present the details and research issues of such a tool, as well as the current state of our prototype.

Original languageEnglish (US)
Title of host publicationFrom Web to Social Web
Subtitle of host publicationDiscovering and Deploying User and Content Profiles - Workshop on Web Mining, WebMine 2006, Revised Selected and Invited Papers
PublisherSpringer Verlag
Number of pages20
ISBN (Print)9783540749509
StatePublished - 2007
Externally publishedYes
EventWorkshop on Web Mining, WebMine 2006 - Berlin, Germany
Duration: Sep 18 2006Sep 18 2006

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume4737 LNAI
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


ConferenceWorkshop on Web Mining, WebMine 2006

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)


Dive into the research topics of 'Extracting and using attribute-value pairs from product descriptions on the Web'. Together they form a unique fingerprint.

Cite this