论文标题
可变度量的迷你批次近端随机递归梯度算法，具有对角线Barzilai-Borwein步骤
A variable metric mini-batch proximal stochastic recursive gradient algorithm with diagonal Barzilai-Borwein stepsize
论文作者
论文摘要
具有不同度量选择的可变度量近端梯度方法已被广泛用于复合优化。将Barzilai-Borwein（BB）方法与对角线选择策略相结合，对角线BB步骤尺寸可以保持较低的每个步骤计算成本，因为标量BB的步骤尺寸并更好地捕获了问题的局部几何形状。在本文中，我们提出了一个可变度量的迷你批次近端随机递归梯度算法VM-MSRGBB，该算法使用新的对角线BB步骤更新度量。 VM-MSRGBB的线性收敛是针对强凸，非凸出和凸功能的。标准数据集的数值实验表明，VM-MSRGBB大于或与某些方差减少的随机梯度方法相媲美，具有最出色的标量量角尺寸或BB步骤尺寸。此外，VM-MSRGBB的性能优于某些高级迷你近端随机梯度方法。
Variable metric proximal gradient methods with different metric selections have been widely used in composite optimization. Combining the Barzilai-Borwein (BB) method with a diagonal selection strategy for the metric, the diagonal BB stepsize can keep low per-step computation cost as the scalar BB stepsize and better capture the local geometry of the problem. In this paper, we propose a variable metric mini-batch proximal stochastic recursive gradient algorithm VM-mSRGBB, which updates the metric using a new diagonal BB stepsize. The linear convergence of VM-mSRGBB is established for strongly convex, non-strongly convex and convex functions. Numerical experiments on standard data sets show that VM-mSRGBB is better than or comparable to some variance reduced stochastic gradient methods with best-tuned scalar stepsizes or BB stepsizes. Furthermore, the performance of VM-mSRGBB is superior to some advanced mini-batch proximal stochastic gradient methods.
