TY - JOUR
T1 - Towards self-exploring discriminating features for visual learning
AU - Wu, Ying
AU - Huang, Thomas S.
N1 - Funding Information:
This work was supported in part by Northwestern Faculty Startup Funds, National Science Foundation Grants CDA-96-24396 and EIA-99-75019 and NSF Alliance Program. The authors also would like to thank the anonymous reviewers for their constructive comments.
Funding Information:
Thomas S. Huang received the B.S. degree in electrical engineering from National Taiwan University, Taipei, Taiwan, China, and the M.S. and Sc.D. degrees in electrical engineering from Massachusetts Institute of Technology (MIT), Cambridge, MA. He was on the Faculty of Department of Electrical Engineering at MIT from 1963 to 1973 and on the Faculty of the School of Electrical Engineering and Director of its Laboratory for Information and Signal Processing at Purdue University from 1973 to 1980. In 1980, he joined the University of Illinois at Urbana-Champaign, where he is now William L. Everitt Distinguished Professor of Electrical and Computer Engineering, Research Professor at the Coordinated Science Laboratory, and Head of the Image Formation and Processing Group at the Beckman Institute for Advanced Science and Technology. During his sabbatical leaves, he has worked at the MIT Lincoln Laboratory, the IBM Thomas J. Watson Research Center, and Rheinishes Landes Museum in Bonn, West Germany, and he held Visiting Professor positions at the Swiss Institutes of Technology in Zurich and Lausanne, the University of Hannover, Germany, INRS-Telecommunications of the University of Quebec, Montreal, Canada, and the University of Tokyo, Japan. He has served as a consultant to numerous industrial firms and government agencies both in the United States and abroad. His professional interests lie in the broad area of information technology, especially the transmission and processing of multidimensional signals. He has published 12 books and over 400 papers in network theory, digital filtering, image processing, and computer vision. He is a Founding Editor of the International Journal Computer Vision , Graphics and Image Processing and Editor of the “Springer Series in Information Sciences”, published by Springer Verlag. Dr. Huang is a Fellow of the International Association of Pattern Recognition, and the Optical Society of American and has received a Guggenheim Fellowship, an A.V. Humboldt Foundation Senior US Scientist Award, and a Fellowship from the Japan Association for the Promotion of Science. He received the IEEE Acoustics, Speech and Signal Processing Society's Technical Achievement Award in 1987 and the Society Award in 1991. He received the IEEE Signal Processing Society's Technical Achievement Award in 1987, and the Society Award in 1991. He was awarded the IEEE Third Millennium Medal in 2000. He received the Honda Lifetime Achievement Award for “contributions to motion analysis” in 2000. He received the IEEE Jack S. Kilby Medal in 2001. Dr. Huang is a member of the National Academy of Engineering, and a Foreign Member of the Chinese Academy of Engineering.
PY - 2002/4
Y1 - 2002/4
N2 - Many visual learning tasks are usually confronted by some common difficulties. One of them is the lack of supervised information, due to the fact that labeling could be tedious, expensive or even impossible. Another difficulty is the high dimensionality of the visual data. Fortunately, these difficulties could be alleviated by using a hybrid of labeled and unlabeled training data for learning. Since the unlabeled data characterize the joint probability across different features, they could be used to boost weak classifiers by exploring discriminating features in a self-supervised fashion. This paper proposes a novel method, the Discriminant-EM (D-EM) algorithm, which attacks these difficulties by integrating discriminant analysis with the EM framework in this hybrid formulation. Both linear and nonlinear methods are investigated in this paper. Based on kernel multiple discriminant analysis, the nonlinear D-EM provides better ability to simplify the probabilistic structures of data distributions in a discrimination space. We also propose a novel data-sampling scheme for efficient learning of kernel discriminants. Our experimental results show that D-EM outperforms a variety of supervised and semi-supervised learning algorithms for many visual learning tasks, such as content-based image retrieval, invariant object recognition, and nonstationary color tracking. The proposed approach could be easily applied for many other learning tasks.
AB - Many visual learning tasks are usually confronted by some common difficulties. One of them is the lack of supervised information, due to the fact that labeling could be tedious, expensive or even impossible. Another difficulty is the high dimensionality of the visual data. Fortunately, these difficulties could be alleviated by using a hybrid of labeled and unlabeled training data for learning. Since the unlabeled data characterize the joint probability across different features, they could be used to boost weak classifiers by exploring discriminating features in a self-supervised fashion. This paper proposes a novel method, the Discriminant-EM (D-EM) algorithm, which attacks these difficulties by integrating discriminant analysis with the EM framework in this hybrid formulation. Both linear and nonlinear methods are investigated in this paper. Based on kernel multiple discriminant analysis, the nonlinear D-EM provides better ability to simplify the probabilistic structures of data distributions in a discrimination space. We also propose a novel data-sampling scheme for efficient learning of kernel discriminants. Our experimental results show that D-EM outperforms a variety of supervised and semi-supervised learning algorithms for many visual learning tasks, such as content-based image retrieval, invariant object recognition, and nonstationary color tracking. The proposed approach could be easily applied for many other learning tasks.
KW - Content-based image retrieval
KW - Discriminant-EM
KW - Nonstationary color tracking
KW - Object recognition
KW - Unlabeled data
KW - Visual learning
UR - http://www.scopus.com/inward/record.url?scp=0036526611&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0036526611&partnerID=8YFLogxK
U2 - 10.1016/S0952-1976(02)00025-8
DO - 10.1016/S0952-1976(02)00025-8
M3 - Article
AN - SCOPUS:0036526611
VL - 15
SP - 139
EP - 150
JO - Engineering Applications of Artificial Intelligence
JF - Engineering Applications of Artificial Intelligence
SN - 0952-1976
IS - 2
ER -