Adapter-Based Parameter-Efficient Adversarial Training for Enhanced Clean Accuracy

Abstract

Adversarial training has emerged as a leading defense against adversarial attacks in deep learning, but suffers from a fundamental robustness-generalization trade-off that can severely degrade clean accuracy. We propose Adapter-based Adversarial Training (Adapter-AT), a novel approach that addresses this critical limitation by leveraging parameter-efficient transfer learning techniques. Our method freezes a clean-trained base model and trains only lightweight adapter modules using adversarial examples, preserving the original clean decision boundaries while adding robustness capabilities. Extensive experiments on MNIST and CIFAR-10 demonstrate that Adapter-AT achieves remarkable clean accuracy improvements of 40.4% and 50.6% respectively compared to standard adversarial training, while maintaining over 98% robustness retention across PGD and AutoAttack evaluations. Our approach requires only 2–3% trainable parameters and provides 73.7% memory reduction as beneficial side effects. These properties make Adapter-AT particularly well-suited for wireless security applications, such as adversarially robust spectrum sensing and signal authentication, where both clean accuracy and computational efficiency are critical constraints. These results establish adapter-based training as an effective solution to the robustness-generalization trade-off, making adversarial training practically viable for applications where clean performance is critical.