In this paper, we compare the BFGS and the conjugate gradient (CG) methods for solving unconstrained problems with a trust-region algorithm. The main result is a new relationship between~CG and the Broyden class, the class of quasi-Newton methods that generalize the BFGS method. This new result allows to rediscover former results established by Broyden in 1970. In addition, we study the use of the limited-memory BFGS (L-BFGS) method in a trust-region algorithm by providing the same properties in comparison with CG. We present numerical results that show a difference of performance between these methods on ill-conditioned and large-scale problems. Some strategies are presented to improve performance by using various amounts of memory and a scaling factor in the L-BFGS method.
Published September 2019 , 34 pages
G1964.pdf (800 KB)