Mobile product image search by automatic query object extraction

Xiaohui Shen, Zhe Lin, Jonathan Brandt, Ying Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

  • 24 Citations

Abstract

Mobile product image search aims at identifying a product, or retrieving similar products from a database based on a photo captured from a mobile phone camera. Application of traditional image retrieval methods (e.g. bag-of-words) to mobile visual search has been shown to be effective in identifying duplicate/near-duplicate photos, near-planar and textured objects such as landmarks, books/cd covers. However, retrieving more general product categories is still a challenging research problem due to variations in viewpoint, illumination, scale, the existence of blur and background clutter in the query image, etc. In this paper, we propose a new approach that can simultaneously extract the product instance from the query, identify the instance, and retrieve visually similar product images. Based on the observation that good query segmentation helps improve retrieval accuracy and good search results provide good priors for segmentation, we formulate our approach in an iterative scheme to improve both query segmentation and retrieval accuracy. To this end, a weighted object mask voting algorithm is proposed based on a spatially-constrained model, which allows robust localization and segmentation of the query object, and achieves significantly better retrieval accuracy than previous methods. We show the effectiveness of our approach by applying it to a large, real-world product image dataset and a new object category dataset.

LanguageEnglish (US)
Title of host publicationComputer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings
Pages114-127
Number of pages14
EditionPART 4
DOIs
StatePublished - Oct 30 2012
Event12th European Conference on Computer Vision, ECCV 2012 - Florence, Italy
Duration: Oct 7 2012Oct 13 2012

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
No.PART 4
Volume7575 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Other

Other12th European Conference on Computer Vision, ECCV 2012
CountryItaly
CityFlorence
Period10/7/1210/13/12

Fingerprint

Image retrieval
Mobile phones
Masks
Lighting
Cameras
Query
Segmentation
Retrieval
Visual Search
Clutter
Image Retrieval
Landmarks
Mobile Phone
Iterative Scheme
Object
Voting
Mask
Illumination
Camera
Cover

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Shen, X., Lin, Z., Brandt, J., & Wu, Y. (2012). Mobile product image search by automatic query object extraction. In Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings (PART 4 ed., pp. 114-127). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7575 LNCS, No. PART 4). DOI: 10.1007/978-3-642-33765-9_9
Shen, Xiaohui ; Lin, Zhe ; Brandt, Jonathan ; Wu, Ying. / Mobile product image search by automatic query object extraction. Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings. PART 4. ed. 2012. pp. 114-127 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 4).
@inproceedings{97bedaa1875f4d40b77705900f287f6e,
title = "Mobile product image search by automatic query object extraction",
abstract = "Mobile product image search aims at identifying a product, or retrieving similar products from a database based on a photo captured from a mobile phone camera. Application of traditional image retrieval methods (e.g. bag-of-words) to mobile visual search has been shown to be effective in identifying duplicate/near-duplicate photos, near-planar and textured objects such as landmarks, books/cd covers. However, retrieving more general product categories is still a challenging research problem due to variations in viewpoint, illumination, scale, the existence of blur and background clutter in the query image, etc. In this paper, we propose a new approach that can simultaneously extract the product instance from the query, identify the instance, and retrieve visually similar product images. Based on the observation that good query segmentation helps improve retrieval accuracy and good search results provide good priors for segmentation, we formulate our approach in an iterative scheme to improve both query segmentation and retrieval accuracy. To this end, a weighted object mask voting algorithm is proposed based on a spatially-constrained model, which allows robust localization and segmentation of the query object, and achieves significantly better retrieval accuracy than previous methods. We show the effectiveness of our approach by applying it to a large, real-world product image dataset and a new object category dataset.",
author = "Xiaohui Shen and Zhe Lin and Jonathan Brandt and Ying Wu",
year = "2012",
month = "10",
day = "30",
doi = "10.1007/978-3-642-33765-9_9",
language = "English (US)",
isbn = "9783642337642",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
number = "PART 4",
pages = "114--127",
booktitle = "Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings",
edition = "PART 4",

}

Shen, X, Lin, Z, Brandt, J & Wu, Y 2012, Mobile product image search by automatic query object extraction. in Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings. PART 4 edn, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), no. PART 4, vol. 7575 LNCS, pp. 114-127, 12th European Conference on Computer Vision, ECCV 2012, Florence, Italy, 10/7/12. DOI: 10.1007/978-3-642-33765-9_9

Mobile product image search by automatic query object extraction. / Shen, Xiaohui; Lin, Zhe; Brandt, Jonathan; Wu, Ying.

Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings. PART 4. ed. 2012. p. 114-127 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 7575 LNCS, No. PART 4).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Mobile product image search by automatic query object extraction

AU - Shen,Xiaohui

AU - Lin,Zhe

AU - Brandt,Jonathan

AU - Wu,Ying

PY - 2012/10/30

Y1 - 2012/10/30

N2 - Mobile product image search aims at identifying a product, or retrieving similar products from a database based on a photo captured from a mobile phone camera. Application of traditional image retrieval methods (e.g. bag-of-words) to mobile visual search has been shown to be effective in identifying duplicate/near-duplicate photos, near-planar and textured objects such as landmarks, books/cd covers. However, retrieving more general product categories is still a challenging research problem due to variations in viewpoint, illumination, scale, the existence of blur and background clutter in the query image, etc. In this paper, we propose a new approach that can simultaneously extract the product instance from the query, identify the instance, and retrieve visually similar product images. Based on the observation that good query segmentation helps improve retrieval accuracy and good search results provide good priors for segmentation, we formulate our approach in an iterative scheme to improve both query segmentation and retrieval accuracy. To this end, a weighted object mask voting algorithm is proposed based on a spatially-constrained model, which allows robust localization and segmentation of the query object, and achieves significantly better retrieval accuracy than previous methods. We show the effectiveness of our approach by applying it to a large, real-world product image dataset and a new object category dataset.

AB - Mobile product image search aims at identifying a product, or retrieving similar products from a database based on a photo captured from a mobile phone camera. Application of traditional image retrieval methods (e.g. bag-of-words) to mobile visual search has been shown to be effective in identifying duplicate/near-duplicate photos, near-planar and textured objects such as landmarks, books/cd covers. However, retrieving more general product categories is still a challenging research problem due to variations in viewpoint, illumination, scale, the existence of blur and background clutter in the query image, etc. In this paper, we propose a new approach that can simultaneously extract the product instance from the query, identify the instance, and retrieve visually similar product images. Based on the observation that good query segmentation helps improve retrieval accuracy and good search results provide good priors for segmentation, we formulate our approach in an iterative scheme to improve both query segmentation and retrieval accuracy. To this end, a weighted object mask voting algorithm is proposed based on a spatially-constrained model, which allows robust localization and segmentation of the query object, and achieves significantly better retrieval accuracy than previous methods. We show the effectiveness of our approach by applying it to a large, real-world product image dataset and a new object category dataset.

UR - http://www.scopus.com/inward/record.url?scp=84867871718&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867871718&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-33765-9_9

DO - 10.1007/978-3-642-33765-9_9

M3 - Conference contribution

SN - 9783642337642

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 114

EP - 127

BT - Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings

ER -

Shen X, Lin Z, Brandt J, Wu Y. Mobile product image search by automatic query object extraction. In Computer Vision, ECCV 2012 - 12th European Conference on Computer Vision, Proceedings. PART 4 ed. 2012. p. 114-127. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); PART 4). Available from, DOI: 10.1007/978-3-642-33765-9_9