论文标题
加入Imagenet上的高精度俱乐部，并使用二进制神经网络票
Join the High Accuracy Club on ImageNet with A Binary Neural Network Ticket
论文作者
论文摘要
二进制神经网络是网络量化的极端情况，长期以来一直认为是潜在的边缘机器学习解决方案。但是，完全精确的同行的明显准确性差距限制了它们在移动应用程序中的创造力。在这项工作中，我们重新审视了二元神经网络的潜力，并专注于引人注目但未解决的问题：二进制神经网络如何在ILSVRC-2012 Imagenet上达到关键的准确性水平（例如80％）？我们通过从三个互补角度增强优化过程来实现这一目标：（1）我们根据对二元架构及其优化过程的全面研究设计了一种新型的二元架构bnext。 （2）我们提出了一种新颖的知识依赖技术，以减轻试图训练极其准确的二进制模型时观察到的违反直觉的过度拟合问题。 （3）我们分析了二进制网络的数据增强管道，并通过完整型模型的最新技术对其进行现代化。 ImageNet上的评估结果表明，Bnext首次将二进制模型精度边界推向80.57％，并显着胜过所有现有的二进制网络。代码和训练有素的模型可在以下网址提供：https：//github.com/hpi-xnor/bnext.git。
Binary neural networks are the extreme case of network quantization, which has long been thought of as a potential edge machine learning solution. However, the significant accuracy gap to the full-precision counterparts restricts their creative potential for mobile applications. In this work, we revisit the potential of binary neural networks and focus on a compelling but unanswered problem: how can a binary neural network achieve the crucial accuracy level (e.g., 80%) on ILSVRC-2012 ImageNet? We achieve this goal by enhancing the optimization process from three complementary perspectives: (1) We design a novel binary architecture BNext based on a comprehensive study of binary architectures and their optimization process. (2) We propose a novel knowledge-distillation technique to alleviate the counter-intuitive overfitting problem observed when attempting to train extremely accurate binary models. (3) We analyze the data augmentation pipeline for binary networks and modernize it with up-to-date techniques from full-precision models. The evaluation results on ImageNet show that BNext, for the first time, pushes the binary model accuracy boundary to 80.57% and significantly outperforms all the existing binary networks. Code and trained models are available at: https://github.com/hpi-xnor/BNext.git.
