论文标题
在分类任务中找到最佳网络深度
Finding the Optimal Network Depth in Classification Tasks
论文作者
论文摘要
我们开发了一种快速的端到端方法，用于使用多个分类器头训练轻型神经网络。通过允许模型确定每个头部的重要性并奖励单个浅层分类器的选择，我们可以检测和删除网络的不需要组件。该操作可以看作是找到模型的最佳深度，大大减少了跨不同硬件处理单元的参数和加速推理的数量，对于许多标准的修剪方法来说，情况并非如此。我们在多个网络体系结构和数据集上显示了方法的性能，分析其优化属性并进行消融研究。
We develop a fast end-to-end method for training lightweight neural networks using multiple classifier heads. By allowing the model to determine the importance of each head and rewarding the choice of a single shallow classifier, we are able to detect and remove unneeded components of the network. This operation, which can be seen as finding the optimal depth of the model, significantly reduces the number of parameters and accelerates inference across different hardware processing units, which is not the case for many standard pruning methods. We show the performance of our method on multiple network architectures and datasets, analyze its optimization properties, and conduct ablation studies.
