论文标题
过度参数线性回归中的本地SGD
Local SGD in Overparameterized Linear Regression
论文作者
论文摘要
我们考虑使用恒定步骤SGD（DSGD）在几个设备上使用Consent Stepize SGD（DSGD）进行分布式学习，每个设备都将最终模型更新发送到中央服务器。在最后一步中，本地估计是汇总的。我们在过度参数化的线性回归一般上限的设置中证明了匹配的下限，并为特定数据生成分布提供了学习率。我们表明，如果局部节点的数量在全球样本量的情况下增长不大，那么多余的风险是差异的顺序。我们进一步将DSGD的样品复杂性与分布式脊回归（DRR）的样品复杂性进行比较，并表明多余的SGD风险小于多余的RR风险，其中两个样品复杂性均具有相同的顺序。
We consider distributed learning using constant stepsize SGD (DSGD) over several devices, each sending a final model update to a central server. In a final step, the local estimates are aggregated. We prove in the setting of overparameterized linear regression general upper bounds with matching lower bounds and derive learning rates for specific data generating distributions. We show that the excess risk is of order of the variance provided the number of local nodes grows not too large with the global sample size. We further compare the sample complexity of DSGD with the sample complexity of distributed ridge regression (DRR) and show that the excess SGD-risk is smaller than the excess RR-risk, where both sample complexities are of the same order.
