Improved New Two-Spectral Conjugate Gradient Iterative Technique for Large Scale Optimization

Abstract

Numerous strategies have been proposed in the field of unconstrained optimization to address various optimization challenges, particularly those associated with large-scale systems. Among the classical methods, Newton and Quasi-Newton approaches are well-known for their rapid convergence properties, especially when initiated with accurate initial estimates. However, these methods rely on second-order derivative information and require the computation or approximation of the Jacobian matrix, which is computationally expensive and thus impractical for large-scale problems. As a result, their direct applicability becomes limited when dealing with high-dimensional optimization tasks.Although originally developed for unconstrained optimization, these methods have been extended to solve systems of nonlinear equations and large-scale optimization problems. In contrast, spectral approaches utilize eigenvalue-based techniques to improve computational efficiency, while conjugate gradient (CG) methods minimize quadratic forms without requiring second-order derivatives, making them more suitable for complex, large-scale systems.The Dai–Yuan (DY) method improves upon the classical CG algorithm by providing a more efficient search direction, which can lead to faster convergence and improved numerical stability in certain scenarios. Similarly, the Hestenes–Stiefel (HS) method offers an alternative formulation that often enhances the convergence rate by more accurately approximating the ideal solution path.More recently, a two-spectral conjugate gradient approach has been explored as a promising technique for solving large-scale unconstrained optimization problems. This method enhances both robustness and efficiency by combining the current gradient direction with a previous search direction, resulting in a more effective search path. Furthermore, many recent CG algorithms incorporate a Wolfe–Powell-type inexact line search strategy, which enables efficient step length determination without the computational burden of exact line search methods. This inexact line search provides a favorable trade-off between computational efficiency and solution accuracy, enhancing the practicality of these methods for large-scale unconstrained optimization