A Fast Approach for Adversarial Training by Fleeing the Illness Parameter Space

Liyao Yin; Shen Wang; Wang Zhenbang; Tian Shigang; Dechen Zhan

doi:10.1007/978-981-99-0605-5_24

A Fast Approach for Adversarial Training by Fleeing the Illness Parameter Space

Liyao Yin
, Shen Wang^*
, Wang Zhenbang
, Tian Shigang
, Dechen Zhan

^*Corresponding author for this work

Faculty of Computing

Harbin Institute of Technology
State Grid Heilongjiang Power Co. Ltd.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Although adversarial training is successful to improve the adversarial robustness under various attacks, the time-consuming is 3–30 times as long as standard training. The main challenge of fast adversarial training is catastrophic overfitting, which breaks the robustness of the model in one training epoch. Although many works have been devoted to solving this problem, a challenging adversarial setting is still not available for these methods. In this paper, we provide a new view of the relationship between the roughness of the adversarial loss and catastrophic overfitting and propose a method with nearly zero cost for times and memories. Our accuracy and robustness can be comparable to the state of the art in the common ϵ adversarial settings. Furthermore, our method can prevent catastrophic overfitting when training and testing under large ϵ adversarial settings, because our method can choose a larger range of hyper-parameters to adapt to the strong adversarial setting.

Original language	English
Title of host publication	Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the 18th IIH-MSP 2022
Editors	Shaowei Weng, Chin-Shiuh Shieh, George A. Tsihrintzis
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	243-253
Number of pages	11
ISBN (Print)	9789819906048
DOIs	https://doi.org/10.1007/978-981-99-0605-5_24
State	Published - 2023
Event	18th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2022 - Kitakyushu, Japan Duration: 16 Dec 2022 → 18 Dec 2022

Publication series

Name	Smart Innovation, Systems and Technologies
Volume	341
ISSN (Print)	2190-3018
ISSN (Electronic)	2190-3026

Conference

Conference	18th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2022
Country/Territory	Japan
City	Kitakyushu
Period	16/12/22 → 18/12/22

Keywords

Adversarial example
Deep learning
Fast Adversarial training
Security

Access to Document

10.1007/978-981-99-0605-5_24

Cite this

Yin, L., Wang, S., Zhenbang, W., Shigang, T., & Zhan, D. (2023). A Fast Approach for Adversarial Training by Fleeing the Illness Parameter Space. In S. Weng, C.-S. Shieh, & G. A. Tsihrintzis (Eds.), Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the 18th IIH-MSP 2022 (pp. 243-253). (Smart Innovation, Systems and Technologies; Vol. 341). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-99-0605-5_24

Yin, Liyao ; Wang, Shen ; Zhenbang, Wang et al. / A Fast Approach for Adversarial Training by Fleeing the Illness Parameter Space. Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the 18th IIH-MSP 2022. editor / Shaowei Weng ; Chin-Shiuh Shieh ; George A. Tsihrintzis. Springer Science and Business Media Deutschland GmbH, 2023. pp. 243-253 (Smart Innovation, Systems and Technologies).

@inproceedings{ee67e7b1e89b4ff8a1a7bea762fad986,

title = "A Fast Approach for Adversarial Training by Fleeing the Illness Parameter Space",

abstract = "Although adversarial training is successful to improve the adversarial robustness under various attacks, the time-consuming is 3–30 times as long as standard training. The main challenge of fast adversarial training is catastrophic overfitting, which breaks the robustness of the model in one training epoch. Although many works have been devoted to solving this problem, a challenging adversarial setting is still not available for these methods. In this paper, we provide a new view of the relationship between the roughness of the adversarial loss and catastrophic overfitting and propose a method with nearly zero cost for times and memories. Our accuracy and robustness can be comparable to the state of the art in the common ϵ adversarial settings. Furthermore, our method can prevent catastrophic overfitting when training and testing under large ϵ adversarial settings, because our method can choose a larger range of hyper-parameters to adapt to the strong adversarial setting.",

keywords = "Adversarial example, Deep learning, Fast Adversarial training, Security",

author = "Liyao Yin and Shen Wang and Wang Zhenbang and Tian Shigang and Dechen Zhan",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.; 18th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2022 ; Conference date: 16-12-2022 Through 18-12-2022",

year = "2023",

doi = "10.1007/978-981-99-0605-5\_24",

language = "英语",

isbn = "9789819906048",

series = "Smart Innovation, Systems and Technologies",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "243--253",

editor = "Shaowei Weng and Chin-Shiuh Shieh and Tsihrintzis, \{George A.\}",

booktitle = "Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the 18th IIH-MSP 2022",

address = "德国",

}

Yin, L, Wang, S, Zhenbang, W, Shigang, T & Zhan, D 2023, A Fast Approach for Adversarial Training by Fleeing the Illness Parameter Space. in S Weng, C-S Shieh & GA Tsihrintzis (eds), Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the 18th IIH-MSP 2022. Smart Innovation, Systems and Technologies, vol. 341, Springer Science and Business Media Deutschland GmbH, pp. 243-253, 18th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2022, Kitakyushu, Japan, 16/12/22. https://doi.org/10.1007/978-981-99-0605-5_24

A Fast Approach for Adversarial Training by Fleeing the Illness Parameter Space. / Yin, Liyao; Wang, Shen; Zhenbang, Wang et al.
Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the 18th IIH-MSP 2022. ed. / Shaowei Weng; Chin-Shiuh Shieh; George A. Tsihrintzis. Springer Science and Business Media Deutschland GmbH, 2023. p. 243-253 (Smart Innovation, Systems and Technologies; Vol. 341).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A Fast Approach for Adversarial Training by Fleeing the Illness Parameter Space

AU - Yin, Liyao

AU - Wang, Shen

AU - Zhenbang, Wang

AU - Shigang, Tian

AU - Zhan, Dechen

PY - 2023

Y1 - 2023

N2 - Although adversarial training is successful to improve the adversarial robustness under various attacks, the time-consuming is 3–30 times as long as standard training. The main challenge of fast adversarial training is catastrophic overfitting, which breaks the robustness of the model in one training epoch. Although many works have been devoted to solving this problem, a challenging adversarial setting is still not available for these methods. In this paper, we provide a new view of the relationship between the roughness of the adversarial loss and catastrophic overfitting and propose a method with nearly zero cost for times and memories. Our accuracy and robustness can be comparable to the state of the art in the common ϵ adversarial settings. Furthermore, our method can prevent catastrophic overfitting when training and testing under large ϵ adversarial settings, because our method can choose a larger range of hyper-parameters to adapt to the strong adversarial setting.

AB - Although adversarial training is successful to improve the adversarial robustness under various attacks, the time-consuming is 3–30 times as long as standard training. The main challenge of fast adversarial training is catastrophic overfitting, which breaks the robustness of the model in one training epoch. Although many works have been devoted to solving this problem, a challenging adversarial setting is still not available for these methods. In this paper, we provide a new view of the relationship between the roughness of the adversarial loss and catastrophic overfitting and propose a method with nearly zero cost for times and memories. Our accuracy and robustness can be comparable to the state of the art in the common ϵ adversarial settings. Furthermore, our method can prevent catastrophic overfitting when training and testing under large ϵ adversarial settings, because our method can choose a larger range of hyper-parameters to adapt to the strong adversarial setting.

KW - Adversarial example

KW - Deep learning

KW - Fast Adversarial training

KW - Security

UR - https://www.scopus.com/pages/publications/85172692875

U2 - 10.1007/978-981-99-0605-5_24

DO - 10.1007/978-981-99-0605-5_24

M3 - 会议稿件

AN - SCOPUS:85172692875

SN - 9789819906048

T3 - Smart Innovation, Systems and Technologies

SP - 243

EP - 253

BT - Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the 18th IIH-MSP 2022

A2 - Weng, Shaowei

A2 - Shieh, Chin-Shiuh

A2 - Tsihrintzis, George A.

PB - Springer Science and Business Media Deutschland GmbH

T2 - 18th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, IIH-MSP 2022

Y2 - 16 December 2022 through 18 December 2022

ER -

Yin L, Wang S, Zhenbang W, Shigang T, Zhan D. A Fast Approach for Adversarial Training by Fleeing the Illness Parameter Space. In Weng S, Shieh CS, Tsihrintzis GA, editors, Advances in Intelligent Information Hiding and Multimedia Signal Processing - Proceeding of the 18th IIH-MSP 2022. Springer Science and Business Media Deutschland GmbH. 2023. p. 243-253. (Smart Innovation, Systems and Technologies). doi: 10.1007/978-981-99-0605-5_24

A Fast Approach for Adversarial Training by Fleeing the Illness Parameter Space

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this