论文标题
修剪人工神经网络：找到良好的高渗透敏锐的最小值的一种方式
Pruning artificial neural networks: a way to find well-generalizing, high-entropy sharp minima
论文作者
论文摘要
最近，朝着深层网络简化的竞赛已经开始，这表明有可能以最小或没有性能损失的最小或没有性能损失来减少这些模型的大小。但是，普遍缺乏理解为什么这些修剪策略有效的原因。在这项工作中，我们将使用两种不同的修剪方法进行比较和分析修剪的解决方案，即一声和逐渐，显示后者的有效性更高。特别是，我们发现逐渐的修剪可以访问狭窄，良好的最小化，通常在使用一击方法时被忽略。在这项工作中，我们还提出了PSP渗透性，这是一种了解给定神经元如何与某些特定学到的类别相关的措施。有趣的是，我们观察到，由迭代模型提取的特征与特定类别的相关性较小，这可能使这些模型更适合转移学习方法。
Recently, a race towards the simplification of deep networks has begun, showing that it is effectively possible to reduce the size of these models with minimal or no performance loss. However, there is a general lack in understanding why these pruning strategies are effective. In this work, we are going to compare and analyze pruned solutions with two different pruning approaches, one-shot and gradual, showing the higher effectiveness of the latter. In particular, we find that gradual pruning allows access to narrow, well-generalizing minima, which are typically ignored when using one-shot approaches. In this work we also propose PSP-entropy, a measure to understand how a given neuron correlates to some specific learned classes. Interestingly, we observe that the features extracted by iteratively-pruned models are less correlated to specific classes, potentially making these models a better fit in transfer learning approaches.
