Improved Three-term Conjugate Gradient Algorithm For Training Neural Network


  • Abbas H. Taqi University of Kirkuk



Feed Forward Neural Network, Training Algorithms, Optimization, Three Terms Conjugate Gradient algorithms Neural Network.


A new three-term conjugate gradient algorithm for training feed-forward neural networks is developed. It is a vector based training algorithm derived from DFP quasi-Newton and has only O(n) memory. The global convergence to the proposed algorithm has been established for convex function under Wolfe condition. The results of numerical experiments are included and compared with other well known training algorithms in this field.


Download data is not yet available.


Haykin S(1994). 'Neural Network A comprehensive function: MacmillanCollege Publishing Company New york".

Rumelhart D., Hinton G., and Williams R (1986).'Learning internal representation by error propagation' . In Rumelhart D and Mcllandeditors, parallel Distributed processing explorations in theMicrostructure of conjugate Cambridge.

Livieris I. and Pintelas P.(2009).' Performance evaluation ofdescent CG methods for neural network training.In E.A. Lipitakis, editor, 9th Hellenic European Research on Computer Mathematic & its Applications Conference (HERCMA’09).

Kalil A. and Hind M. (2014). 'Conjugate gradient algorithm based on Aiten'sProcessfor training neural networks'. Raf. J. of Comp & Math. Vol.(11), No.(1). DOI:

Battiti R. (1992). '1st and 2nd order method for learning betweensteepest descent and Newton method'. Neural Comp. 4(2). DOI:

Fletcher R. and Reeves C. (1964).' function minimization byconjugate gradients'. Computer Journal. (7). DOI:

Noccedal J.(1996).' conjugate gradient methods and nonlinearoptimization. In [Adams.J.L.Nazareth (EDS.) linear and nonlinear conjugate gradient related methods. SIAM.

Parker D.(1987).'Optimal algorithms for adaptive Networks:second order back- propagation, second order Direct propagation,and second order Hebian learning.' Proceedings of the 1987 IEEE International Conference on Neural Networks, vol (2).

Andreas A. and Wu S. (2007).'Practical Optimization Algorithms and Engineering Applications', Springer Science and Business Media, LLC New York.

Livieris I., and pintelas P.(2011).'An advanced conjugate gradientraining algorithm based on a modified secant equation'. TechnicalReport No. TR11-03'. University of patras. Dep.of Math. GR-265 04 patras , Greece.

Andrei N.(2013). 'A simple three-term conjugate gradient f unconstrained optimization'. J. of Computation and AppliedMathematics (241). DOI:

Zoutendijk G. (1970). 'Non-linear programming computational Methods', in J. Abadi (Ed). Integer and Non-linear Programming,North Holland.

Nguyen D. and Widrow B. (1990).' Improving the learning speedof 2-layer neural network by choosing initial values of adaptiveweights. Biological Cybernetics, 59. DOI:

Shanno D. and Phua K.(1976).' Minimization of unconstrained multivariate functions'. ACM Transactions on Mathematical Software, 2. DOI:




How to Cite

Taqi, A. H. (2015). Improved Three-term Conjugate Gradient Algorithm For Training Neural Network. Journal of Kufa for Mathematics and Computer, 2(3), 93–100.

Similar Articles

1 2 3 4 5 6 > >> 

You may also start an advanced similarity search for this article.