论文标题
具有应用的梯度方法的三维二次终止的机制
A mechanism of three-dimensional quadratic termination for the gradient method with applications
论文作者
论文摘要
最近的研究表明，二维二次终止特性在改善梯度方法的性能方面具有巨大的潜力。但是，尚不清楚高维二次终止是否会带来进一步的好处。在本文中，我们通过引入梯度方法的三维二次终止机制来提供肯定的答案。一个新的步骤尺寸来自该机制，使得与新的步骤尺寸的延迟梯度方法家族具有三维二次终止特性。当应用于barzilai-borwein（BB）方法时，新颖的步骤不需要使用任何确切的线路搜索或Hessian，并且可以通过先前迭代中的步骤和梯度规范来计算。使用长BB步骤和与新颖步骤相关的一些短步骤，我们开发了一种有效的梯度方法来进行二次优化，并将其进一步扩展到一般不受约束的优化。数值实验表明，三维二次终止属性可以显着提高BB方法的性能，而所提出的方法优于使用二维二次二次终止属性的梯度方法。
Recent studies show that the two-dimensional quadratic termination property has great potential in improving performance of the gradient method. However, it is not clear whether higher-dimensional quadratic termination leads further benefits. In this paper, we provide an affirmative answer by introducing a mechanism of three-dimensional quadratic termination for the gradient method. A novel stepsize is derived from the mechanism such that a family of delayed gradient methods equipping with the novel stepsize have the three-dimensional quadratic termination property. When applied to the Barzilai--Borwein (BB) method, the novel stepsize does not require the use of any exact line search or the Hessian, and can be computed by stepsizes and gradient norms in previous iterations. Using long BB steps and some short steps associated with the novel stepsize in an adaptive manner, we develop an efficient gradient method for quadratic optimization and further extend it to general unconstrained optimization. Numerical experiments show that the three-dimensional quadratic termination property can significantly improve performance of the BB method, and the proposed method outperforms gradient methods that use stepsizes with the two-dimensional quadratic termination property.
