Semantic preserving distance metric learning and applications

How do we accurately browse a large set of images or efficiently annotate the images from an image library? Image clustering methods are invaluable tools for applications such as content-based image retrieval and image annotation. To perform these tasks, it is critical to have proper features to describe the visual and semantic content of images and to define an accurate distance metric to measure the dissimilarity between any two images. However, existing methods, which adopt the features of color histograms, edge direction histograms and shape context, lack the ability to describe semantic content. To solve this problem, we propose a new approach that utilizes user-provided pairwise constraints to describe the semantic relationship between two images. A Semantic Preserving Distance Metric Learning (SP-DML) algorithm is developed to explore the complementary characteristics of the visual features and pairwise constraints in a unified feature space. In this space, the learned distance metric can be used to measure the dissimilarity between two images. Specifically, the manifold structure adopted in SP-DML is revealed by the image's visual features. To integrate semantic contents in distance metric learning, SP-DML utilizes pairwise constraints to build semantic patches and align these patches to obtain the optimal distance metric for the new feature space. Experimental results in image clustering demonstrate that the performance of SP-DML is appealing.