Volume 26 , Issue 2 , PP: 258-278, 2025 | Cite this article as | XML | Html | PDF | Full Length Article
Ahmad Alhawarat 1 , Sultanah Masmali 2 , Ibrahim M. Sulaiman 3 , Issam A. R. Moghrabi 4 * , Norazura Ahmad 5 , Shahrina Ismail 6
Doi: https://doi.org/10.54216/IJNS.260220
In recent years, there has been a surge of attention to the Conjugate Gradient Method (CGM) and its applications. This is because the algorithm of CGM does not require the computation of the second derivative or an approximation during the iteration process. In this study, a four-term descent CGM is proposed by utilizing the famous Polak–Ribiere–Polyak (PRP) conjugate gradient formula. The direction of the proposed method achieves the descent property without line search consideration. In addition, the convergence properties are met to generate the stationary points. Findings from numerical experiments on unconstrained optimization and robotic motion control problems demonstrate that the novel approach outperforms some existing methods including the famous CG-Descent conjugate gradient method.
Inexact line search , Conjugate gradient method , Descent condition , CG-Descent , Numerical comparison
[1] P. Wolfe, "Convergence conditions for ascent methods," SIAM Review, vol. 11, pp. 226–235, 1969.
[2] P. Wolfe, "Convergence conditions for ascent methods. II: Some corrections," SIAM Review, vol. 13, pp. 185–188, 1971.
[3] M. R. Hestenes and E. Stiefel, "Methods of conjugate gradients for solving linear systems," J. Res. Natl. Bur. Stand., vol. 49, pp. 409–436, 1952.
[4] E. Polak and G. Ribiere, "Note sur la convergence de méthodes de directions conjuguées," Rev. Française d’Informatique et de Recherche Opérationnelle. Série Rouge, vol. 3, pp. 35–43, 1969.
[5] Y. Liu and C. Storey, "Efficient generalized conjugate gradient algorithms, part 1: Theory," J. Optim. Theory Appl., vol. 69, pp. 129–137, 1991.
[6] R. Fletcher, "Function minimization by conjugate gradients," Comput. J., vol. 7, pp. 149–154, 1964.
[7] R. Fletcher, Practical Methods of Optimization, 2nd ed. New York: Wiley, 2000.
[8] Y.-H. Dai and Y. Yuan, "A nonlinear conjugate gradient method with a strong global convergence property," SIAM J. Optim., vol. 10, pp. 177–182, 1999.
[9] Y.-H. Dai and L.-Z. Liao, "New conjugacy conditions and related nonlinear conjugate gradient methods," Appl. Math. Optim., vol. 43, pp. 87–101, 2001.
[10] W. W. Hager and H. Zhang, "Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent," ACM Trans. Math. Softw., vol. 32, no. 1, pp. 113–137, 2006.
[11] W. W. Hager and H. Zhang, "The limited memory conjugate gradient method," SIAM J. Optim., vol. 23, pp. 2150–2168, 2013.
[12] N. Andrei, "A simple three-term conjugate gradient algorithm for unconstrained optimization," J. Comput. Appl. Math., vol. 241, pp. 19–29, 2013.
[13] G. Zhou, Y. Yang, M. Cao, and S. Qu, "A new spectral three-term conjugate gradient method with random parameter based on modified secant equation and its application to low-carbon supply chain optimization," J. Math., vol. 2022, Art. ID 8939770, 2022.
[14] S. Babaie-Kafaki and R. Ghanbari, "Two modified three-term conjugate gradient methods with sufficient descent property," Optim. Lett., vol. 8, pp. 2285–2297, 2014.
[15] S. Deng and Z. Wan, "A three-term conjugate gradient algorithm for large-scale unconstrained optimization problems," Appl. Numer. Math., vol. 92, pp. 70–81, 2015.
[16] J.-K. Liu, J.-Ł. Xu, and L.-Q. Zhang, "Partially symmetrical derivative-free Liu–Storey projection method for convex constrained equations," Int. J. Comput. Math., vol. 96, pp. 1787–1798, 2019.
[17] J.-K. Liu, Y.-X. Zhao, and X.-L. Wu, "Some three-term conjugate gradient methods with the new direction structure," Appl. Numer. Math., vol. 150, pp. 433–443, 2020.
[18] A. Alhawarat, G. Alhamzi, I. Masmali, and Z. Salleh, "A descent four-term conjugate gradient method with global convergence properties for large-scale unconstrained optimisation problems," Math. Probl. Eng., vol. 2021, Art. ID 6651234, 2021.
[19] A. Alhawarat, Z. Salleh, and I. A. Masmali, "A convex combination between two different search directions of conjugate gradient method and application in image restoration," Math. Probl. Eng., vol. 2021, Art. ID 6697025, 2021.
[20] J. C. Gilbert and J. Nocedal, "Global convergence properties of conjugate gradient methods for optimization," SIAM J. Optim., vol. 2, pp. 21–42, 1992.
[21] N. Andrei, "An unconstrained optimization test functions collection," Adv. Model. Optim., vol. 10, pp. 147–161, 2008.
[22] E. D. Dolan and J. J. Moré, "Benchmarking optimization software with performance profiles," Math. Program., vol. 91, pp. 201–213, 2002.
[23] Z. Salleh, G. Alhamzi, I. Masmali, and A. Alhawarat, "A modified Liu and Storey conjugate gradient method for large scale unconstrained optimization problems," Algorithms, vol. 14, no. 1, Art. ID 18, 2021.
[24] M. Al-Baali, "Descent property and global convergence of the Fletcher–Reeves method with inexact line search," IMA J. Numer. Anal., vol. 5, pp. 121–124, 1985.
[25] G. Zoutendijk, "Nonlinear programming, computational methods," in Integer and Nonlinear Programming, J. Abadie, Ed. Amsterdam: North-Holland, 1970, pp. 37–86.
[26] S. Sra, S. Nowozin, and S. J. Wright, Eds., Optimization for Machine Learning. Cambridge, MA: MIT Press, 2012.
[27] N. Andrei, "A modified Polak–Ribière–Polyak conjugate gradient algorithm for unconstrained optimization," Optim., vol. 60, no. 12, pp. 1457–1471, 2011.
[28] H. Mohammad, I. M. Sulaiman, and M. Mamat, "Two diagonal conjugate gradient-like methods for unconstrained optimization," J. Ind. Manag. Optim., vol. 20, no. 1, pp. 170–187, 2024.
[29] E. G. Birgin and J. M. Martinez, "A spectral conjugate gradient method for unconstrained optimization," Appl. Math. Optim., vol. 43, pp. 117–128, 2001.
[30] Y. Narushima and H. Yabe, "Conjugate gradient methods based on secant conditions that generate descent search directions for unconstrained optimization," J. Comput. Appl. Math., vol. 236, pp. 4303–4317, 2012.
[31] Y. Zhang et al., "General four-step discrete-time zeroing and derivative dynamics applied to time-varying nonlinear optimization," J. Comput. Appl. Math., vol. 347, pp. 314–329, 2019.
[32] M. M. Yahaya et al., "New Generalized Quasi-Newton Algorithm," IEEE Access, vol. 10, pp. 10816–10826, 2022.
[33] R. B. Yunus et al., "A Modified Structured Spectral HS Method," Mathematics, vol. 11, no. 14, Art. ID 3215, 2023.
