论文标题
深神网络的渐近软簇修剪
Asymptotic Soft Cluster Pruning for Deep Neural Networks
论文作者
论文摘要
过滤器修剪方法通过去除选定的过滤器来引入结构稀疏性，因此对于降低复杂性特别有效。以前的作品从经验修剪网络从较小规范的过滤器的角度来看，对最终结果的贡献较小。但是，此类标准已被证明对过滤器的分布敏感，并且由于修剪后的容量差距是固定的，因此准确性可能很难恢复。在本文中，我们提出了一种称为渐近软簇修剪（ASCP）的新型过滤器修剪方法，以根据过滤器的相似性来识别网络的冗余。首先，来自参数过度的网络的每个过滤器都是通过聚类来区分的，然后重建以手动将冗余引入其中。提出了一些聚类指南，以更好地保留特征提取能力。重建后，允许更新过滤器，以消除错误选择的效果。此外，还采用了各种修剪率的衰减策略来稳定修剪过程并改善最终性能。通过在每个群集中逐渐生成更相同的过滤器，ASCP可以通过通道添加操作将它们删除，而几乎没有准确性下降。关于CIFAR-10和Imagenet数据集的广泛实验表明，与许多最新算法相比，我们的方法可以取得竞争成果。
Filter pruning method introduces structural sparsity by removing selected filters and is thus particularly effective for reducing complexity. Previous works empirically prune networks from the point of view that filter with smaller norm contributes less to the final results. However, such criteria has been proven sensitive to the distribution of filters, and the accuracy may hard to recover since the capacity gap is fixed once pruned. In this paper, we propose a novel filter pruning method called Asymptotic Soft Cluster Pruning (ASCP), to identify the redundancy of network based on the similarity of filters. Each filter from over-parameterized network is first distinguished by clustering, and then reconstructed to manually introduce redundancy into it. Several guidelines of clustering are proposed to better preserve feature extraction ability. After reconstruction, filters are allowed to be updated to eliminate the effect caused by mistakenly selected. Besides, various decaying strategies of the pruning rate are adopted to stabilize the pruning process and improve the final performance as well. By gradually generating more identical filters within each cluster, ASCP can remove them through channel addition operation with almost no accuracy drop. Extensive experiments on CIFAR-10 and ImageNet datasets show that our method can achieve competitive results compared with many state-of-the-art algorithms.
