论文标题
使用多任务高斯过程找到不均匀的量化方案
Finding Non-Uniform Quantization Schemes using Multi-Task Gaussian Processes
论文作者
论文摘要
我们提出了一种用于神经网络量化的新方法，该方法将神经体系结构搜索问题施加了，作为在CNN层中找到不均匀的位分布的高参数搜索之一。我们在先前的多任务高斯进程中执行搜索，该过程将问题拆分为多个任务，每个任务都与不同数量的训练时期相对应，并通过对那些产生最大信息的配置来探索空间。然后，我们证明，在最后一层中，精度明显降低，我们可以通过可观的记忆节省来最小的准确性损失。我们使用VGG，Resnet和Googlenet架构在CIFAR10和Imagenet数据集上测试我们的发现。
We propose a novel method for neural network quantization that casts the neural architecture search problem as one of hyperparameter search to find non-uniform bit distributions throughout the layers of a CNN. We perform the search assuming a Multi-Task Gaussian Processes prior, which splits the problem to multiple tasks, each corresponding to different number of training epochs, and explore the space by sampling those configurations that yield maximum information. We then show that with significantly lower precision in the last layers we achieve a minimal loss of accuracy with appreciable memory savings. We test our findings on the CIFAR10 and ImageNet datasets using the VGG, ResNet and GoogLeNet architectures.
