论文标题
Push-saga：一种分散的随机算法，其有向图的方差降低
Push-SAGA: A decentralized stochastic algorithm with variance reduction over directed graphs
论文作者
论文摘要
在本文中，我们提出了Push-Saga，这是一种分散的随机一阶方法，用于在有针对性的节点网络上最小化有限和最小化。 Push-Saga结合了节点级别差异的降低，以消除由随机梯度，网络级梯度跟踪引起的不确定性，以解决数据的分布式性质以及推动和 - 共识，以应对有向通信链接的挑战。我们表明，Push-Saga将线性收敛到精确溶液，以解决平滑和强烈凸出问题，因此是在任意强烈连接的有向图上的第一个线性构造的随机算法。我们还表征了Push-Saga与其集中式同行相比实现线性加速并实现与网络无关的收敛速率相比的状态。我们在强烈凸和非凸问题上的数值实验的帮助下说明了Push-Saga的行为和收敛性。
In this paper, we propose Push-SAGA, a decentralized stochastic first-order method for finite-sum minimization over a directed network of nodes. Push-SAGA combines node-level variance reduction to remove the uncertainty caused by stochastic gradients, network-level gradient tracking to address the distributed nature of the data, and push-sum consensus to tackle the challenge of directed communication links. We show that Push-SAGA achieves linear convergence to the exact solution for smooth and strongly convex problems and is thus the first linearly-convergent stochastic algorithm over arbitrary strongly connected directed graphs. We also characterize the regimes in which Push-SAGA achieves a linear speed-up compared to its centralized counterpart and achieves a network-independent convergence rate. We illustrate the behavior and convergence properties of Push-SAGA with the help of numerical experiments on strongly convex and non-convex problems.
