Conditional random fields and its application to language analysis system

Label bias problem of discriminative models has negative effects on sequential labeling. Conditional random fields were proposed to solve label bias problem based on the normalization of the sequential probability. Through the transformation from language analysis to sequential labeling, the language analysis system, including segmentation, part-of-speech tagging and chunking, was built based on conditional random fields and feature selection. Experimental results show that conditional random fields outperform other discriminative models in the language analysis system and overcome label bias problem.