The aim of this project is to investigate a unified framework of learning kernel machines via the integration of several novel machine learning methodologies as well as the smart exploitation of the power of human computation. In particular, a number of novel machine learning techniques will be investigated in this project, including semi-supervised kernel learning, fast training of supervised kernel machines, active learning, and distance metric learning, etc. We have already proposed some novel algorithms in several sub-tasks, including non-parametric kernel (NPK) learning algorithms for kernel learning, batch mode active learning (BMAL) for selecting informative unlabeled examples, and various algorithms of distance metric learning for solving some real-world challenges.
　

One key challenge to content-based multimedia retrieval is known as the semantic gap issue, which exists between the low-level visual features and the high-level semantic concepts. To enhance the retrieval performance of multimedia retrieval, including image and video shot retrieval, an important tool is to design an interactive multimedia retrieval engine, which enables the computer system to understand the users' search intent by learning with users' feedbacks. The interactive multimedia retrieval technique is a powerful tool to reduce the semantic gap challenge and hence can significantly improve the retrieval performance in practice. In this project, we are investigating novel machine learning techniques for solving the relevance feedbacks in content-based image and retrieval. The key research challenges include: 1) how to learn with small-sized training data, which is the case in relevance feedback, 2) how to select a subset of most informative examples, which is known as the batch mode active learning, etc.
　

In order to tackle the semantic gap challenge of multimedia retrieval in a long-term consideration, we have proposed the scheme of Collaborative Multimedia Retrieval (CMR), which explores statistical machine learning techniques in mining users' log data of relevance feedback for the retrieval tasks. In the CMR project, we suggest an online relevance feedback scheme with users’ historical logs, termed log-based relevance feedback (LRF) techniques. On the other hand, to exploit users' logs more effectively, we propose an offline distance metric learning (DML) scheme to learn a robust metrics from user logs for bridging the semantic gap in multimedia retrieval [9]. We believe that many techniques studied in the CMR project are essential to tackling many challenges of personalization in multimedia search and information retrieval.
 

Face technologies are important for many real-world applications. The aim of this project is to investigate new technologies of face detection, recognition and annotation for varied applications in multimedia, computer vision, and augmented reality. We have investigated some novel face techniques for some multimedia applications, especially for solving automatic face annotation problems. We are also developing intelligent techniques for exploring rich media data on WWW with face and machine learning technologies.
　

5. Cross-Media and Cross-Language Multimedia Retrieval

1) Video over InternEt and Wireless Technologies (iVIEW): This was a large project done in CUHK, which was collaborated with CMU Infomedia project. I had studied video processing technologies for Hong Kong News video data processing and developed novel techniques for video similarity matching, video copy detection, and cross-media video search.

2) Multi-Layer Video Content Classification and Retrieval: This project developed effective machine learning algorithms for conducting video content classification and multimodal search on Hong Kong News video data. We proposed a multi-layer framework for classifying the video contents, which will then be used for helping the search and browsing tasks.

3) The Cross-Language Image Retrieval Track (ImageCLEF): I have participated in the activities of the CLEF image retrieval tracks. We have investigated effective techniques and solutions for solving some challenges, including cross-language retrieval, content-based visual ranking, and multi-modal fusion of multiple modalities. 

4) TREC Video Retrieval Track (TRECVID): We have participated in the activities of TREC video retrieval, especially in  two tasks: shot boundary detection and automatic video search in TRECVID before. We have suggested the novel multimodal and multilevel ranking framework for large-scale video retrieval and have developed effective algorithms for solving the challenges.
　

The World Wide Web (WWW) has become one of the biggest digital media repository, which has growth very rapidly in recent years. This project aims to explore novel techniques for solving the Web media search and mining tasks. In particular, we are investigating novel techniques for mining WWW images, photos, videos, and other rich media contents, from popular social websites, including Flickr, YouTube, etc.
　

Building an accurate and efficient classifier is crucial to making a handwriting recognizer practical for real-world mobile computing environment. In this project, we study a framework of LEarning To Effectively Recognize handwriting, “LETER” for short, for building highly accurate and efficient handwriting recognizers in mobile computing applications. In particular, we will investigate kernel machine learning methods together with active learning techniques for building accurate and efficient classifiers for handwriting recognizers. Two major research challenges will be addressed in this project. One is to how to build an effective kernel classifier for handwriting recognizers. The other is how to find the most informative examples for labeling in order to build a compact yet very accurate classifier for mobile computing applications.