论文标题
网络修剪的过滤草图
Filter Sketch for Network Pruning
论文作者
论文摘要
我们通过信息保存预训练的网络权重（过滤器）提出了一种新型的网络修剪方法。使用信息保存的网络修剪作为矩阵草图问题，可以通过现成的频繁方向方法有效地解决。我们的方法（称为filtersketch）编码了预训练的权重的二阶信息，这使得可以通过简单的微调过程恢复修剪的网络的表示能力。 Filtersketch既不需要从头开始训练，也不需要数据驱动的迭代优化，从而导致修剪优化的时间成本降低了数量的数量。 CIFAR-10上的实验表明，过滤器可降低63.3％的flops和prunes的网络参数的59.9％的网络参数，而RESNET-1110的精确度可忽略不计。在ILSVRC-2012上，它减少了45.5％的拖鞋，并删除了43.0％的参数，而Resnet-50的准确度仅为0.69％。可以在https://github.com/lmbxmu/filtersketch上找到我们的代码和修剪模型。
We propose a novel network pruning approach by information preserving of pre-trained network weights (filters). Network pruning with the information preserving is formulated as a matrix sketch problem, which is efficiently solved by the off-the-shelf Frequent Direction method. Our approach, referred to as FilterSketch, encodes the second-order information of pre-trained weights, which enables the representation capacity of pruned networks to be recovered with a simple fine-tuning procedure. FilterSketch requires neither training from scratch nor data-driven iterative optimization, leading to a several-orders-of-magnitude reduction of time cost in the optimization of pruning. Experiments on CIFAR-10 show that FilterSketch reduces 63.3% of FLOPs and prunes 59.9% of network parameters with negligible accuracy cost for ResNet-110. On ILSVRC-2012, it reduces 45.5% of FLOPs and removes 43.0% of parameters with only 0.69% accuracy drop for ResNet-50. Our code and pruned models can be found at https://github.com/lmbxmu/FilterSketch.
