Skip to main navigation Skip to search Skip to main content

AGAE: Unsupervised Anomaly Detection for Encrypted Malicious Traffic

  • Hao Wang
  • , Ye Wang*
  • , Zhaoquan Gu
  • , Yan Jia
  • *Corresponding author for this work
  • Harbin Institute of Technology Shenzhen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Nowadays, In order to protect the privacy and security of network users, network traffic is extensively encrypted. However, encrypted traffic can also be exploited by attackers to conceal their malicious activities. Moreover, existing approaches heavily rely on supervised learning and labeled datasets. Thus, effectively detecting malicious traffic with limited data remains an unresolved issue. In this paper, we propose AGAE, an unsupervised anomaly detection system for detecting malicious traffic, based on the Attribute Graph AutoEncoder we designed. We innovatively analyze the convergence and reusability of network attacks, and design AGAE by using these two characteristics. We conduct extensive experiments to evaluate the performance of AGAE. The experimental results illustrate that the AGAE achieves average AUC of 0.961. And the average F1 achieves 0.974, which outperform the state-of-the-art methods. In particular, AGAE has stronger detection capabilities against traditional brute force attacks and encrypted flooding traffic.

Original languageEnglish
Title of host publicationWeb and Big Data - 8th International Joint Conference, APWeb-WAIM 2024, Proceedings
EditorsWenjie Zhang, Zhengyi Yang, Xiaoyang Wang, Anthony Tung, Zhonglong Zheng, Hongjie Guo
PublisherSpringer Science and Business Media Deutschland GmbH
Pages448-464
Number of pages17
ISBN (Print)9789819772407
DOIs
StatePublished - 2024
Externally publishedYes
Event8th Asia-Pacific Web and Web-Age Information Management Joint International Conference on Web and Big Data, APWeb-WAIM 2024 - Jinhua, China
Duration: 30 Aug 20241 Sep 2024

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14964 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference8th Asia-Pacific Web and Web-Age Information Management Joint International Conference on Web and Big Data, APWeb-WAIM 2024
Country/TerritoryChina
CityJinhua
Period30/08/241/09/24

Keywords

  • Anomaly detection
  • Autoencoder
  • Malicious Traffic detection

Fingerprint

Dive into the research topics of 'AGAE: Unsupervised Anomaly Detection for Encrypted Malicious Traffic'. Together they form a unique fingerprint.

Cite this