论文标题
解释通过摊销的推断显着图来处理网络修剪
Interpretations Steered Network Pruning via Amortized Inferred Saliency Maps
论文作者
论文摘要
卷积神经网络（CNN）压缩对于在资源有限的边缘设备中部署这些模型至关重要。 CNN的现有通道修剪算法在复杂模型上取得了很大的成功。他们从各个角度解决了修剪问题，并使用不同的指标来指导修剪过程。但是，这些指标主要集中于模型的“输出”或“权重”，而忽略了其“解释”信息。为了填补这一空白，我们建议通过利用模型的解释来指导修剪过程，从而从新颖的角度解决通道修剪问题，从而利用来自模型的输入和输出的信息。但是，现有的解释方法无法部署以实现我们的目标，因为它们的修剪效率低下，或者可能预测了非固定解释。我们通过引入选择器模型来解决这一挑战，该模型可以预测修剪模型的实时平滑显着性掩码。我们通过径向基函数（RBF）函数来参数化解释性掩码的分布，以在我们选择器模型的电感偏置中纳入自然图像的几何事物。因此，我们可以获得解释的紧凑表示，以降低修剪方法的计算成本。我们利用我们的选择器模型来指导网络修剪，以最大程度地提高修剪和原始模型的解释性表示的相似性。关于CIFAR-10和Imagenet基准数据集的广泛实验证明了我们提出的方法的功效。我们的实现可在\ url {https://github.com/alii-ganjj/interpretationssteerperpruning}中获得
Convolutional Neural Networks (CNNs) compression is crucial to deploying these models in edge devices with limited resources. Existing channel pruning algorithms for CNNs have achieved plenty of success on complex models. They approach the pruning problem from various perspectives and use different metrics to guide the pruning process. However, these metrics mainly focus on the model's `outputs' or `weights' and neglect its `interpretations' information. To fill in this gap, we propose to address the channel pruning problem from a novel perspective by leveraging the interpretations of a model to steer the pruning process, thereby utilizing information from both inputs and outputs of the model. However, existing interpretation methods cannot get deployed to achieve our goal as either they are inefficient for pruning or may predict non-coherent explanations. We tackle this challenge by introducing a selector model that predicts real-time smooth saliency masks for pruned models. We parameterize the distribution of explanatory masks by Radial Basis Function (RBF)-like functions to incorporate geometric prior of natural images in our selector model's inductive bias. Thus, we can obtain compact representations of explanations to reduce the computational costs of our pruning method. We leverage our selector model to steer the network pruning by maximizing the similarity of explanatory representations for the pruned and original models. Extensive experiments on CIFAR-10 and ImageNet benchmark datasets demonstrate the efficacy of our proposed method. Our implementations are available at \url{https://github.com/Alii-Ganjj/InterpretationsSteeredPruning}
