论文标题
分析具有平滑激活函数的完全连接深神经网络回归估计的收敛速率
Analysis of the rate of convergence of fully connected deep neural network regression estimates with smooth activation function
论文作者
论文摘要
本文有助于当前的深神经网络（DNNS）的统计理论。结果表明，如果适当地限制了对回归函数的结构，则DNN能够规避所谓的维度诅咒。在大多数结果中，调谐参数是网络的稀疏性，它描述了网络中非零权重的数量。该约束似乎是收敛结果良好率的关键因素。最近，假设被驳回。特别是，表明简单的完全连接的DNN可以达到相同的收敛速率。这些完全连接的DNN基于未绑定的Relu激活函数。在本文中，我们将结果扩展到平滑的激活函数，即，到Sigmoid激活函数。结果表明，基于完全连接的具有Sigmoid激活函数的完全连接的DNN的估计器也达到了最小收敛速率（最多$ \ ln n $ factors）。在我们的结果中，隐藏层的数量是固定的，每层神经元的数量倾向于无穷大，以实现无限限，并且给出了网络中的权重。
This article contributes to the current statistical theory of deep neural networks (DNNs). It was shown that DNNs are able to circumvent the so--called curse of dimensionality in case that suitable restrictions on the structure of the regression function hold. In most of those results the tuning parameter is the sparsity of the network, which describes the number of non-zero weights in the network. This constraint seemed to be the key factor for the good rate of convergence results. Recently, the assumption was disproved. In particular, it was shown that simple fully connected DNNs can achieve the same rate of convergence. Those fully connected DNNs are based on the unbounded ReLU activation function. In this article we extend the results to smooth activation functions, i.e., to the sigmoid activation function. It is shown that estimators based on fully connected DNNs with sigmoid activation function also achieve the minimax rates of convergence (up to $\ln n$-factors). In our result the number of hidden layers is fixed, the number of neurons per layer tends to infinity for sample size tending to infinity and a bound for the weights in the network is given.
