Chinese text chunking based on improved K-means clustering

An improved k-means clustering method is proposed to identify Chinese phrases with the purpose of avoiding data sparseness and taking think of the relationship of neighbor part of speech and the cohesion of all part of speeches within one phrase. The proposed method regards each phrase as a cluster whose kernel is headword, which richly used the constituent disciplinarian of one phrase. It also integrates supervised statistical method and unsupervised clustering method by setting the original center of each class according the data from small Chinese corpus, which not only improves the accuracy of clustering but also avoids data sparseness. Through testing on Chinese Penn Treebank, the F score of seven types of Chinese phrase achieves to 92.94%. So, it is effective for Chinese text chunking.