A Quantitative Approach for Threat Assessment from Heterogeneous Web Security Logs

Intrusion detection systems (IDS) and web application firewalls (WAF) generate overwhelming volumes of alerts, yet lack fine-grained threat assessment capabilities to identify valuable alert clues or high-risk threat events from them. Regarding this problem, we propose a Regression-based fine-grained Threat Assessment Framework called RegTAF for evaluating and analyzing threats in multi-source Web security logs. RegTAF extracts threat entities (e.g., security events and malicious IPs) and characterizes them along multiple dimensions. To overcome label scarcity and severe class imbalance in security logs, we employ unsupervised representation learning to enhance feature discriminability. A regression model is then trained to output the assessment of threat severity, enabling precise risk quantification for security events. Through ablation studies and comparative experiments across multiple regression algorithms, we validate the effectiveness of RegTAF, identify the key threat dimensions governing severity assessment, and provide complexity analysis with different regression algorithms. By implementing RegTAF, high-threat or potentially harmful threat entities can be selected based on fine-grained threat severity, thereby improving the efficiency of web attack detection analysis.