Fast adversarial training based on implicit sharpness aware regularization

Deep neural networks (DNNs) are highly vulnerable to adversarial examples. While adversarial training (AT) is the most effective defense, it faces a severe efficiency-robustness trade-off: multi-step methods like Projected Gradient Descent (PGD) are robust but computationally expensive, whereas fast single-step alternatives like Fast Gradient Sign Method (FGSM) suffer from catastrophic overfitting (CO). In this paper, we identify sharp minima in the loss landscape as a primary driver of CO. To address this, we propose Model Smoothing (MS), an implicit sharpness-aware regularization framework that suppresses sharp minima via architectural smoothing. Unlike explicit methods (e.g., Sharpness-Aware Minimization) that require two forward and backward passes, our approach achieves implicit regularization through architectural modifications – injecting zero-mean noise into convolutional layers and replacing ReLU with a tunable smooth activation (β-SiLU) – with no additional backward passes. Extensive experiments on CIFAR-10, SVHN, CIFAR-100, and ImageNet demonstrate that MS achieves competitive robust accuracy (e.g., 49.18% under ϵ=8/255 against PGD-50 on CIFAR-10, 20.11% on ImageNet) while training nearly three times faster than PGD-based methods. AutoAttack evaluation confirms the robustness is genuine, not artifacts of gradient masking.