Discovering thematic objects in image collections and videos

Junsong Yuan; Gangqiang Zhao; Yun Fu; Zhu Li; Aggelos K. Katsaggelos; Ying Wu

doi:10.1109/TIP.2011.2181952

Discovering thematic objects in image collections and videos

Junsong Yuan^*, Gangqiang Zhao, Yun Fu, Zhu Li, Aggelos K. Katsaggelos, Ying Wu

^*Corresponding author for this work

Electrical and Computer Engineering

Research output: Contribution to journal › Article › peer-review

32 Scopus citations

Abstract

Given a collection of images or a short video sequence, we define a thematic object as the key object that frequently appears and is the representative of the visual contents. Successful discovery of the thematic object is helpful for object search and tagging, video summarization and understanding, etc. However, this task is challenging because 1) there lacks a priori knowledge of the thematic objects, such as their shapes, scales, locations, and times of re-occurrences, and 2) the thematic object of interest can be under severe variations in appearances due to viewpoint and lighting condition changes, scale variations, etc. Instead of using a top-down generative model to discover thematic visual patterns, we propose a novel bottom-up approach to gradually prune uncommon local visual primitives and recover the thematic objects. A multilayer candidate pruning procedure is designed to accelerate the image data mining process. Our solution can efficiently locate thematic objects of various sizes and can tolerate large appearance variations of the same thematic object. Experiments on challenging image and video data sets and comparisons with existing methods validate the effectiveness of our method.

Original language	English (US)
Article number	6112717
Pages (from-to)	2207-2219
Number of pages	13
Journal	IEEE Transactions on Image Processing
Volume	21
Issue number	4
DOIs	https://doi.org/10.1109/TIP.2011.2181952
State	Published - Apr 2012

Funding

Manuscript received August 05, 2010; revised February 06, 2011, July 18, 2011 and December 03, 2011; accepted December 03, 2011. Date of publication December 26, 2011; date of current version March 21, 2012. This work was supported in part by the Nanyang Assistant Professorship (SUG M5804001) to Dr. J. Yuan. Dr. Y. Fu was supported in part by Futurewei (Huawei) Technologies Inc. and the Intelligence Community Postdoctoral Research Fellowship under Award 2011-11071400006. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Xuelong Li.

Keywords

Image data mining
thematic object discovery

ASJC Scopus subject areas

Software
Computer Graphics and Computer-Aided Design

Access to Document

10.1109/TIP.2011.2181952

Cite this

@article{85cc67404574482d8d304af2e3633d03,

title = "Discovering thematic objects in image collections and videos",

abstract = "Given a collection of images or a short video sequence, we define a thematic object as the key object that frequently appears and is the representative of the visual contents. Successful discovery of the thematic object is helpful for object search and tagging, video summarization and understanding, etc. However, this task is challenging because 1) there lacks a priori knowledge of the thematic objects, such as their shapes, scales, locations, and times of re-occurrences, and 2) the thematic object of interest can be under severe variations in appearances due to viewpoint and lighting condition changes, scale variations, etc. Instead of using a top-down generative model to discover thematic visual patterns, we propose a novel bottom-up approach to gradually prune uncommon local visual primitives and recover the thematic objects. A multilayer candidate pruning procedure is designed to accelerate the image data mining process. Our solution can efficiently locate thematic objects of various sizes and can tolerate large appearance variations of the same thematic object. Experiments on challenging image and video data sets and comparisons with existing methods validate the effectiveness of our method.",

keywords = "Image data mining, thematic object discovery",

author = "Junsong Yuan and Gangqiang Zhao and Yun Fu and Zhu Li and Katsaggelos, {Aggelos K.} and Ying Wu",

note = "Funding Information: Manuscript received August 05, 2010; revised February 06, 2011, July 18, 2011 and December 03, 2011; accepted December 03, 2011. Date of publication December 26, 2011; date of current version March 21, 2012. This work was supported in part by the Nanyang Assistant Professorship (SUG M5804001) to Dr. J. Yuan. Dr. Y. Fu was supported in part by Futurewei (Huawei) Technologies Inc. and the Intelligence Community Postdoctoral Research Fellowship under Award 2011-11071400006. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Xuelong Li.",

year = "2012",

month = apr,

doi = "10.1109/TIP.2011.2181952",

language = "English (US)",

volume = "21",

pages = "2207--2219",

journal = "IEEE Transactions on Image Processing",

issn = "1057-7149",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "4",

}

TY - JOUR

T1 - Discovering thematic objects in image collections and videos

AU - Yuan, Junsong

AU - Zhao, Gangqiang

AU - Fu, Yun

AU - Li, Zhu

AU - Katsaggelos, Aggelos K.

AU - Wu, Ying

N1 - Funding Information: Manuscript received August 05, 2010; revised February 06, 2011, July 18, 2011 and December 03, 2011; accepted December 03, 2011. Date of publication December 26, 2011; date of current version March 21, 2012. This work was supported in part by the Nanyang Assistant Professorship (SUG M5804001) to Dr. J. Yuan. Dr. Y. Fu was supported in part by Futurewei (Huawei) Technologies Inc. and the Intelligence Community Postdoctoral Research Fellowship under Award 2011-11071400006. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Xuelong Li.

PY - 2012/4

Y1 - 2012/4

N2 - Given a collection of images or a short video sequence, we define a thematic object as the key object that frequently appears and is the representative of the visual contents. Successful discovery of the thematic object is helpful for object search and tagging, video summarization and understanding, etc. However, this task is challenging because 1) there lacks a priori knowledge of the thematic objects, such as their shapes, scales, locations, and times of re-occurrences, and 2) the thematic object of interest can be under severe variations in appearances due to viewpoint and lighting condition changes, scale variations, etc. Instead of using a top-down generative model to discover thematic visual patterns, we propose a novel bottom-up approach to gradually prune uncommon local visual primitives and recover the thematic objects. A multilayer candidate pruning procedure is designed to accelerate the image data mining process. Our solution can efficiently locate thematic objects of various sizes and can tolerate large appearance variations of the same thematic object. Experiments on challenging image and video data sets and comparisons with existing methods validate the effectiveness of our method.

AB - Given a collection of images or a short video sequence, we define a thematic object as the key object that frequently appears and is the representative of the visual contents. Successful discovery of the thematic object is helpful for object search and tagging, video summarization and understanding, etc. However, this task is challenging because 1) there lacks a priori knowledge of the thematic objects, such as their shapes, scales, locations, and times of re-occurrences, and 2) the thematic object of interest can be under severe variations in appearances due to viewpoint and lighting condition changes, scale variations, etc. Instead of using a top-down generative model to discover thematic visual patterns, we propose a novel bottom-up approach to gradually prune uncommon local visual primitives and recover the thematic objects. A multilayer candidate pruning procedure is designed to accelerate the image data mining process. Our solution can efficiently locate thematic objects of various sizes and can tolerate large appearance variations of the same thematic object. Experiments on challenging image and video data sets and comparisons with existing methods validate the effectiveness of our method.

KW - Image data mining

KW - thematic object discovery

UR - http://www.scopus.com/inward/record.url?scp=84859024517&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84859024517&partnerID=8YFLogxK

U2 - 10.1109/TIP.2011.2181952

DO - 10.1109/TIP.2011.2181952

M3 - Article

C2 - 22207639

AN - SCOPUS:84859024517

SN - 1057-7149

VL - 21

SP - 2207

EP - 2219

JO - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

IS - 4

M1 - 6112717

ER -

Discovering thematic objects in image collections and videos

Abstract

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this