论文标题
检查和减轻横杆非理想性对稀疏深神经网络的准确实施的影响
Examining and Mitigating the Impact of Crossbar Non-idealities for Accurate Implementation of Sparse Deep Neural Networks
论文作者
论文摘要
最近，已经引入了几种结构化修剪技术，以实施具有较少数量的横杆的深神经网络（DNN）。尽管这些技术声称可以保留横梁上稀疏DNN的准确性，但没有人研究了不可阻挡的横杆非理想性对修剪网络实际性能的影响。为此，我们进行了一项全面的研究，以表明与映射到非理想的横梁上的未经修复的DNN相比，这会导致显着的横杆压缩率高度稀疏的DNN会导致严重的精度损失。我们使用具有基准数据集（CIFAR10和CIFAR100）的VGG11和VGG16 DNN上的多种结构化旋转方法（例如，C/F修剪，XCS和XRS）进行实验。我们提出了两种缓解方法 - 横杆柱重排和重量受限训练（WCT） - 可以与稀疏DNNS的横杆映射集成，以最大程度地减少修剪模型产生的准确性损失。这些有助于通过增加横杆上低电导突触的比例来减轻非理想性，从而提高其计算精度。
Recently several structured pruning techniques have been introduced for energy-efficient implementation of Deep Neural Networks (DNNs) with lesser number of crossbars. Although, these techniques have claimed to preserve the accuracy of the sparse DNNs on crossbars, none have studied the impact of the inexorable crossbar non-idealities on the actual performance of the pruned networks. To this end, we perform a comprehensive study to show how highly sparse DNNs, that result in significant crossbar-compression-rate, can lead to severe accuracy losses compared to unpruned DNNs mapped onto non-ideal crossbars. We perform experiments with multiple structured-pruning approaches (such as, C/F pruning, XCS and XRS) on VGG11 and VGG16 DNNs with benchmark datasets (CIFAR10 and CIFAR100). We propose two mitigation approaches - Crossbar column rearrangement and Weight-Constrained-Training (WCT) - that can be integrated with the crossbar-mapping of the sparse DNNs to minimize accuracy losses incurred by the pruned models. These help in mitigating non-idealities by increasing the proportion of low conductance synapses on crossbars, thereby improving their computational accuracies.
