Improved feature selection approach TFIDF in text mining

Li Ping Jing; Hou Kuan Huang; Hong Bo Shi

Improved feature selection approach TFIDF in text mining

Li Ping Jing^*
, Hou Kuan Huang
, Hong Bo Shi

^*Corresponding author for this work

Beijing Jiaotong University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

214 Scopus citations

Abstract

This paper describes one Feature Selection method (TFIDF). With it, we process the data resource and set up the VSM model in order to provide a convenient data structure for text categorization. We calculate the precision of this method with the help of categorization results. According to the empirical results, we analyze its advantages and disadvantages and present a new TFIDF-based feature selection approach to improve its accuracy.

Original language	English
Title of host publication	2002 International Conference on Machine Learning and Cybernetics, ICMLC 2002
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	944-946
Number of pages	3
ISBN (Print)	0780375084, 9780780375086
State	Published - 2002
Externally published	Yes
Event	2002 International Conference on Machine Learning and Cybernetics, ICMLC 2002 - Beijing, China Duration: 4 Nov 2002 → 5 Nov 2002

Publication series

Name	Proceedings of 2002 International Conference on Machine Learning and Cybernetics
Volume	2

Conference

Conference	2002 International Conference on Machine Learning and Cybernetics, ICMLC 2002
Country/Territory	China
City	Beijing
Period	4/11/02 → 5/11/02

Keywords

Evaluation function
Feature selection
TFIDF
Text mining
VSM

Cite this

@inproceedings{aed69c138e0947f7ad30a9282366beb1,

title = "Improved feature selection approach TFIDF in text mining",

abstract = "This paper describes one Feature Selection method (TFIDF). With it, we process the data resource and set up the VSM model in order to provide a convenient data structure for text categorization. We calculate the precision of this method with the help of categorization results. According to the empirical results, we analyze its advantages and disadvantages and present a new TFIDF-based feature selection approach to improve its accuracy.",

keywords = "Evaluation function, Feature selection, TFIDF, Text mining, VSM",

author = "Jing, \{Li Ping\} and Huang, \{Hou Kuan\} and Shi, \{Hong Bo\}",

year = "2002",

language = "英语",

isbn = "0780375084",

series = "Proceedings of 2002 International Conference on Machine Learning and Cybernetics",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "944--946",

booktitle = "2002 International Conference on Machine Learning and Cybernetics, ICMLC 2002",

address = "美国",

note = "2002 International Conference on Machine Learning and Cybernetics, ICMLC 2002 ; Conference date: 04-11-2002 Through 05-11-2002",

}

Jing, LP, Huang, HK & Shi, HB 2002, Improved feature selection approach TFIDF in text mining. in 2002 International Conference on Machine Learning and Cybernetics, ICMLC 2002. Proceedings of 2002 International Conference on Machine Learning and Cybernetics, vol. 2, Institute of Electrical and Electronics Engineers Inc., pp. 944-946, 2002 International Conference on Machine Learning and Cybernetics, ICMLC 2002, Beijing, China, 4/11/02.

Improved feature selection approach TFIDF in text mining. / Jing, Li Ping; Huang, Hou Kuan; Shi, Hong Bo.
2002 International Conference on Machine Learning and Cybernetics, ICMLC 2002. Institute of Electrical and Electronics Engineers Inc., 2002. p. 944-946 (Proceedings of 2002 International Conference on Machine Learning and Cybernetics; Vol. 2).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Improved feature selection approach TFIDF in text mining

AU - Jing, Li Ping

AU - Huang, Hou Kuan

AU - Shi, Hong Bo

PY - 2002

Y1 - 2002

N2 - This paper describes one Feature Selection method (TFIDF). With it, we process the data resource and set up the VSM model in order to provide a convenient data structure for text categorization. We calculate the precision of this method with the help of categorization results. According to the empirical results, we analyze its advantages and disadvantages and present a new TFIDF-based feature selection approach to improve its accuracy.

AB - This paper describes one Feature Selection method (TFIDF). With it, we process the data resource and set up the VSM model in order to provide a convenient data structure for text categorization. We calculate the precision of this method with the help of categorization results. According to the empirical results, we analyze its advantages and disadvantages and present a new TFIDF-based feature selection approach to improve its accuracy.

KW - Evaluation function

KW - Feature selection

KW - TFIDF

KW - Text mining

KW - VSM

UR - https://www.scopus.com/pages/publications/0036929622

M3 - 会议稿件

AN - SCOPUS:0036929622

SN - 0780375084

SN - 9780780375086

T3 - Proceedings of 2002 International Conference on Machine Learning and Cybernetics

SP - 944

EP - 946

BT - 2002 International Conference on Machine Learning and Cybernetics, ICMLC 2002

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2002 International Conference on Machine Learning and Cybernetics, ICMLC 2002

Y2 - 4 November 2002 through 5 November 2002

ER -

Improved feature selection approach TFIDF in text mining

Abstract

Publication series

Conference

Keywords

Other files and links

Fingerprint

Cite this