论文标题
深度relu编程
Deep ReLU Programming
论文作者
论文摘要
前馈恢复神经网络将其输入结构域划分为有限的许多神经元激活模式和仿射行为的“仿射区域”。我们分析了它们的数学结构，并提供算法原始素，以有效地应用与线性编程相关的技术，以迭代最小化此类非凸功能的功能。特别是，我们提出了单纯形算法的扩展，该算法在诱导的顶点上迭代，但此外，还能够在相邻的“仿射区域”上有效地更改其可行区域。这样，我们将获得LAD回归的Barrodale-Roberts算法作为一种特殊情况，但也能够在每一步中训练L1训练损失的第一层神经网络。
Feed-forward ReLU neural networks partition their input domain into finitely many "affine regions" of constant neuron activation pattern and affine behaviour. We analyze their mathematical structure and provide algorithmic primitives for an efficient application of linear programming related techniques for iterative minimization of such non-convex functions. In particular, we propose an extension of the Simplex algorithm which is iterating on induced vertices but, in addition, is able to change its feasible region computationally efficiently to adjacent "affine regions". This way, we obtain the Barrodale-Roberts algorithm for LAD regression as a special case, but also are able to train the first layer of neural networks with L1 training loss decreasing in every step.
