论文标题
随机亚级别方案的高概率边界，具有沉重的尾巴噪声
High Probability Bounds for Stochastic Subgradient Schemes with Heavy Tailed Noise
论文作者
论文摘要
在这项工作中，我们研究了沉重的尾巴噪声下随机亚级别方法的高概率界限。在这种情况下，仅假定噪声具有有限的方差，而不是次高斯的分布，众所周知，标准亚级别方法具有很高的概率边界。我们分析了投影的随机亚级别方法的剪裁版本，其中每当具有较大规范时，亚级别的估计值将被截断。我们表明，这种剪裁策略既导致了几乎最佳的任何时间和许多经典平均方案的有限范围。表明初步实验可支持该方法的有效性。
In this work we study high probability bounds for stochastic subgradient methods under heavy tailed noise. In this setting the noise is only assumed to have finite variance as opposed to a sub-Gaussian distribution for which it is known that standard subgradient methods enjoys high probability bounds. We analyzed a clipped version of the projected stochastic subgradient method, where subgradient estimates are truncated whenever they have large norms. We show that this clipping strategy leads both to near optimal any-time and finite horizon bounds for many classical averaging schemes. Preliminary experiments are shown to support the validity of the method.
