论文标题
基于知识蒸馏的中国语法错误校正
Chinese grammatical error correction based on knowledge distillation
论文作者
论文摘要
鉴于攻击测试集和大型模型参数上现有的中国语法误差校正模型的鲁棒性差，本文使用知识蒸馏的方法来压缩模型参数并提高模型的反攻击能力。在数据方面，攻击测试集是通过将干扰集成到标准评估数据集中来构建的，并且通过攻击测试集评估模型鲁棒性。实验结果表明，蒸馏小型模型可以确保在减少模型参数数量的条件下确保性能并提高训练速度，并在攻击测试集中实现最佳效果，并且鲁棒性得到显着提高。代码可从https://github.com/richard888888/kd-cgec获得。
In view of the poor robustness of existing Chinese grammatical error correction models on attack test sets and large model parameters, this paper uses the method of knowledge distillation to compress model parameters and improve the anti-attack ability of the model. In terms of data, the attack test set is constructed by integrating the disturbance into the standard evaluation data set, and the model robustness is evaluated by the attack test set. The experimental results show that the distilled small model can ensure the performance and improve the training speed under the condition of reducing the number of model parameters, and achieve the optimal effect on the attack test set, and the robustness is significantly improved. Code is available at https://github.com/Richard88888/KD-CGEC.
