Beyond Hard Samples: Robust and Effective Grammatical Error Correction with Cycle Self-Augmenting

Recent studies have revealed that grammatical error correction methods in the sequence-to-sequence paradigm are vulnerable to adversarial attacks. Large Language Models (LLMs) also inevitably experience decreased performance when confronted with adversarial examples, even for the GPT-3.5 model. In this paper, we propose a simple yet very effective Cycle Self-Augmenting (CSA) method and conduct a thorough robustness evaluation of cutting-edge GEC methods with three different types of adversarial attacks. By leveraging the augmenting data generated from the GEC models themselves in the post-training stage and introducing regularization data for cycle training, our proposed method can effectively improve the model robustness of well-trained GEC models with only a few more training epochs at an extra cost. Experiments indicate that our proposed training method can significantly enhance the robustness and performance of the state-of-the-art GEC model.