Deep click feature based query merging for robust image recognition

We address the problem of image representation with user click data, wherein each image is represented as a count vector based on its clicked queries. As the query set obtained from search engines is large-scale and redundant, this image representation is extremely high-dimensional and with low discriminative ability. To deal with this issue, we propose a deep click feature based query clustering approach, and construct a compact and low-dimensional click feature with merged queries. Specially, to learn the deep click feature, we construct a smooth image-click graph instead of the direct image-click vector to represent each query, and use it as the input of the convolutional network. A similarity graph based re-sorting and propagation method is applied to construct the click graph. We evaluate our method on the public Clickture-Dog dataset. Experimental results show that: 1) Query merging with image-click graph outperforms that with image-click vector, since it improves the click-unbalance among categories and captures more structured information; 2) The deep model helps to generate a powerful hierarchical click feature for queries, making an improved clustering result.