论文标题
强大神经网络的功能误差校正
Functional Error Correction for Robust Neural Networks
论文作者
论文摘要
当在硬件中实现神经网络(神经网络)时,它们的权重需要存储在内存设备中。随着噪声在存储的重量中积累,神经网的性能将降低。本文研究了如何使用错误校正代码(ECC)来保护权重。与数据存储中的经典误差校正不同,优化目标是在错误校正后优化神经网络的性能,而不是最大程度地减少受保护位中不可纠正的位错误率。也就是说,通过将神经网络视为其输入的函数,误差校正方案是面向功能的。一个主要的挑战是,深神经网络通常具有数亿至数亿至数亿的重量,从而为ECC造成了较大的冗余开销,并且权重及其神经网络的性能之间的关系可能非常复杂。为了应对挑战,我们提出了一个选择性保护(SP)方案,该方案仅选择一个重要的ECC保护位。为了找到此类位,并在ECC的冗余和NeuralNet的表现之间实现了优化的权衡,我们提出了一种基于深厚的增强学习的算法。实验结果验证了与天然基线方案相比,所提出的算法在功能误差校正任务方面的性能要好得多。
When neural networks (NeuralNets) are implemented in hardware, their weights need to be stored in memory devices. As noise accumulates in the stored weights, the NeuralNet's performance will degrade. This paper studies how to use error correcting codes (ECCs) to protect the weights. Different from classic error correction in data storage, the optimization objective is to optimize the NeuralNet's performance after error correction, instead of minimizing the Uncorrectable Bit Error Rate in the protected bits. That is, by seeing the NeuralNet as a function of its input, the error correction scheme is function-oriented. A main challenge is that a deep NeuralNet often has millions to hundreds of millions of weights, causing a large redundancy overhead for ECCs, and the relationship between the weights and its NeuralNet's performance can be highly complex. To address the challenge, we propose a Selective Protection (SP) scheme, which chooses only a subset of important bits for ECC protection. To find such bits and achieve an optimized tradeoff between ECC's redundancy and NeuralNet's performance, we present an algorithm based on deep reinforcement learning. Experimental results verify that compared to the natural baseline scheme, the proposed algorithm achieves substantially better performance for the functional error correction task.