论文标题
具有动量的近端梯度算法和灵活参数重新启动，以进行非convex优化
Proximal Gradient Algorithm with Momentum and Flexible Parameter Restart for Nonconvex Optimization
论文作者
论文摘要
已经提出了各种类型的参数重新启动方案，用于加速梯度算法，以促进其在凸优化中的实际收敛。但是，在非convex优化中，参数重新启动下加速梯度算法的收敛属性仍然晦涩难懂。在本文中，我们提出了一种新型的加速近端梯度算法，该梯度算法具有参数重新启动（命名为APG-Restart），用于解决非凸和非平滑问题。我们的APG-RESTART设计为1）允许采用涵盖许多现有计划的灵活参数重新启动方案； 2）在非凸和非滑动优化的全局子线性收敛速率中； 3）保证收敛到临界点，并取决于非凸和非滑动优化的局部几何形状的参数化，具有多种类型的渐近收敛速率。数值实验证明了我们提出的算法的有效性。
Various types of parameter restart schemes have been proposed for accelerated gradient algorithms to facilitate their practical convergence in convex optimization. However, the convergence properties of accelerated gradient algorithms under parameter restart remain obscure in nonconvex optimization. In this paper, we propose a novel accelerated proximal gradient algorithm with parameter restart (named APG-restart) for solving nonconvex and nonsmooth problems. Our APG-restart is designed to 1) allow for adopting flexible parameter restart schemes that cover many existing ones; 2) have a global sub-linear convergence rate in nonconvex and nonsmooth optimization; and 3) have guaranteed convergence to a critical point and have various types of asymptotic convergence rates depending on the parameterization of local geometry in nonconvex and nonsmooth optimization. Numerical experiments demonstrate the effectiveness of our proposed algorithm.
