Information-theoretic agglomerative K-means

Agglomerative K-means is a clustering algorithm of K-means type. The algorithm has good properties because of its insensitiveness to the locations of initial centers and its effectiveness in determining the number of clusters. In present study, we extend the agglomerative K-means from information theoretic view and develop a new clustering algorithm, Information-Theoretic Agglomerative K-means. Different from the agglomerative K-means, we propose a new objective function employing the Kullback-Leibler divergence to measure the dispersion of clusters. Based on this objective function, we derive the updating formulas of centers and membership for objects associated to different centers and then develop an efficient algorithm. Experimental results on both well-separated and overlapped data suggested that the proposed clustering algorithm is not only promising in obtaining good clustering performance but also effective in identifying the number of clusters.