论文标题
通过稀疏实现对抗性鲁棒性
Achieving Adversarial Robustness via Sparsity
论文作者
论文摘要
已知网络修剪会产生紧凑的模型，而不会准确地降解。但是，修剪过程如何影响网络的鲁棒性和背后的工作机制仍未解决。在这项工作中，我们从理论上证明网络权重的稀疏性与模型鲁棒性密切相关。通过对各种对抗修剪方法的实验，我们发现权重稀疏性不会受到伤害，而是改善稳健性，在这种情况下，这两种权重都从彩票票和对抗性训练中继承来改善网络修剪的模型稳健性。基于这些发现，我们提出了一种称为逆权重继承的新型对抗训练方法，该方法通过从小型网络继承权重，在大网络上施加了稀疏的权重分布，从而提高了大型网络的稳健性。
Network pruning has been known to produce compact models without much accuracy degradation. However, how the pruning process affects a network's robustness and the working mechanism behind remain unresolved. In this work, we theoretically prove that the sparsity of network weights is closely associated with model robustness. Through experiments on a variety of adversarial pruning methods, we find that weights sparsity will not hurt but improve robustness, where both weights inheritance from the lottery ticket and adversarial training improve model robustness in network pruning. Based on these findings, we propose a novel adversarial training method called inverse weights inheritance, which imposes sparse weights distribution on a large network by inheriting weights from a small network, thereby improving the robustness of the large network.
