论文标题
CNN层的大规范不会伤害对抗性的鲁棒性
Large Norms of CNN Layers Do Not Hurt Adversarial Robustness
论文作者
论文摘要
Since the Lipschitz properties of convolutional neural networks (CNNs) are widely considered to be related to adversarial robustness, we theoretically characterize the $\ell_1$ norm and $\ell_\infty$ norm of 2D multi-channel convolutional layers and provide efficient methods to compute the exact $\ell_1$ norm and $\ell_\infty$ norm.基于我们的定理，我们提出了一种称为规范衰减的新型正则化方法，该方法可以有效地降低卷积层和完全连接的层的规范。实验表明，规范指定方法，包括规范衰减，重量衰减和奇异值剪切，可以改善CNN的概括。但是，它们可能会稍微伤害对抗性的鲁棒性。观察到这种意外现象，我们计算了CNN中使用三个不同的对抗训练框架训练的CNN层规范，并且令人惊讶地发现，对抗性强大的CNN比非对抗性稳健的同类产品具有可比性甚至更大的层规范。此外，我们证明，在一个温和的假设下，可以使用神经网络来实现对抗性稳健的分类器，并且对抗性稳健的神经网络可以具有任意较大的Lipschitz常数。因此，在CNN层上执行小规范既不是必要的也不是有效实现对抗性鲁棒性的。该代码可从https://github.com/youweiliang/norm_robustness获得。
Since the Lipschitz properties of convolutional neural networks (CNNs) are widely considered to be related to adversarial robustness, we theoretically characterize the $\ell_1$ norm and $\ell_\infty$ norm of 2D multi-channel convolutional layers and provide efficient methods to compute the exact $\ell_1$ norm and $\ell_\infty$ norm. Based on our theorem, we propose a novel regularization method termed norm decay, which can effectively reduce the norms of convolutional layers and fully-connected layers. Experiments show that norm-regularization methods, including norm decay, weight decay, and singular value clipping, can improve generalization of CNNs. However, they can slightly hurt adversarial robustness. Observing this unexpected phenomenon, we compute the norms of layers in the CNNs trained with three different adversarial training frameworks and surprisingly find that adversarially robust CNNs have comparable or even larger layer norms than their non-adversarially robust counterparts. Furthermore, we prove that under a mild assumption, adversarially robust classifiers can be achieved using neural networks, and an adversarially robust neural network can have an arbitrarily large Lipschitz constant. For this reason, enforcing small norms on CNN layers may be neither necessary nor effective in achieving adversarial robustness. The code is available at https://github.com/youweiliang/norm_robustness.
