论文标题
关于随机梯度下降的扩散极限的注释
A note on diffusion limits for stochastic gradient descent
论文作者
论文摘要
在机器学习文献中，随机梯度下降最近因其所谓的隐式正规化特性而被广泛讨论。试图阐明噪声在随机梯度算法中的作用的大部分理论，它们通过具有高斯噪声的随机微分方程被广泛近似的随机梯度下降。我们为这种实践提供了一种新颖的严格理论理由，展示了噪声的高斯如何自然产生。
In the machine learning literature stochastic gradient descent has recently been widely discussed for its purported implicit regularization properties. Much of the theory, that attempts to clarify the role of noise in stochastic gradient algorithms, has widely approximated stochastic gradient descent by a stochastic differential equation with Gaussian noise. We provide a novel rigorous theoretical justification for this practice that showcases how the Gaussianity of the noise arises naturally.
