论文标题
使用混合整数编程识别ANN体系结构中的关键神经元
Identifying Critical Neurons in ANN Architectures using Mixed Integer Programming
论文作者
论文摘要
我们介绍了一个混合整数程序（MIP），以在深神网络体系结构中为每个神经元分配重要性得分，这是由其同时修剪网络主要学习任务的影响的指导。通过仔细设计MIP的目标函数，我们驱动求解器以最大程度地减少需要保留的关键神经元数（即具有很高的重要性评分），以保持训练有素的神经网络的整体准确性。此外，提出的配方通过识别多个“幸运”子网络来概括最近考虑的彩票优化，从而导致了优化的体系结构，这些体系结构不仅在单个数据集中表现良好，而且在重新培训网络权重时会在多个数据集中进行概括。最后，我们通过使用辅助网络将跨层的重要性分数解耦来提供可扩展的方法的可扩展实现。我们证明了我们的制定能力修剪神经网络，其准确性和对流行数据集和体系结构的概括性的损失边缘损失。
We introduce a mixed integer program (MIP) for assigning importance scores to each neuron in deep neural network architectures which is guided by the impact of their simultaneous pruning on the main learning task of the network. By carefully devising the objective function of the MIP, we drive the solver to minimize the number of critical neurons (i.e., with high importance score) that need to be kept for maintaining the overall accuracy of the trained neural network. Further, the proposed formulation generalizes the recently considered lottery ticket optimization by identifying multiple "lucky" sub-networks resulting in optimized architecture that not only performs well on a single dataset, but also generalizes across multiple ones upon retraining of network weights. Finally, we present a scalable implementation of our method by decoupling the importance scores across layers using auxiliary networks. We demonstrate the ability of our formulation to prune neural networks with marginal loss in accuracy and generalizability on popular datasets and architectures.
