论文标题
通过神经网络对BV功能的近似：一种规律性理论方法
Approximation of BV functions by neural networks: A regularity theory approach
论文作者
论文摘要
在本文中，我们关注的是单个隐藏层神经网络在单位圆上具有relu激活函数的函数。特别是，我们对数据点数超过节点数量的情况感兴趣。我们首先研究了与成本函数相关的随机梯度流的平衡，并进行了二次惩罚。具体而言，我们证明了庞加莱的不平等，用于具有独立于数据和节点数量的明确常数的惩罚版本。随着我们的惩罚偏向要限制的权重，这使我们研究了一个有界权重的网络可以近似界限变化（BV）的函数。 关于BV函数近似的我们的主要贡献是我们称之为本地化定理的结果。具体而言，它指出，相对于不受约束的问题（全局最佳最佳），权重小于$ r $的预期误差是$ r^{ - 1/9} $。该主题中的证明是新颖的，受到椭圆形部分微分方程理论的技术的启发。最后，我们通过证明通用近似定理的定量版本来量化全局最佳的期望值。
In this paper we are concerned with the approximation of functions by single hidden layer neural networks with ReLU activation functions on the unit circle. In particular, we are interested in the case when the number of data-points exceeds the number of nodes. We first study the convergence to equilibrium of the stochastic gradient flow associated with the cost function with a quadratic penalization. Specifically, we prove a Poincaré inequality for a penalized version of the cost function with explicit constants that are independent of the data and of the number of nodes. As our penalization biases the weights to be bounded, this leads us to study how well a network with bounded weights can approximate a given function of bounded variation (BV). Our main contribution concerning approximation of BV functions, is a result which we call the localization theorem. Specifically, it states that the expected error of the constrained problem, where the length of the weights are less than $R$, is of order $R^{-1/9}$ with respect to the unconstrained problem (the global optimum). The proof is novel in this topic and is inspired by techniques from regularity theory of elliptic partial differential equations. Finally we quantify the expected value of the global optimum by proving a quantitative version of the universal approximation theorem.
