Improved Three-term Conjugate Gradient Algorithm For Training Neural Network

Authors

  • Abbas H. Taqi University of Kirkuk

DOI:

https://doi.org/10.31642/JoKMC/2018/020309

Keywords:

Feed Forward Neural Network, Training Algorithms, Optimization, Three Terms Conjugate Gradient algorithms Neural Network.

Abstract

A new three-term conjugate gradient algorithm for training feed-forward neural networks is developed. It is a vector based training algorithm derived from DFP quasi-Newton and has only O(n) memory. The global convergence to the proposed algorithm has been established for convex function under Wolfe condition. The results of numerical experiments are included and compared with other well known training algorithms in this field.

Downloads

Download data is not yet available.

References

Haykin S(1994). 'Neural Network A comprehensive function: MacmillanCollege Publishing Company New york".

Rumelhart D., Hinton G., and Williams R (1986).'Learning internal representation by error propagation' . In Rumelhart D and Mcllandeditors, parallel Distributed processing explorations in theMicrostructure of conjugate Cambridge.

Livieris I. and Pintelas P.(2009).' Performance evaluation ofdescent CG methods for neural network training.In E.A. Lipitakis, editor, 9th Hellenic European Research on Computer Mathematic & its Applications Conference (HERCMA’09).

Kalil A. and Hind M. (2014). 'Conjugate gradient algorithm based on Aiten'sProcessfor training neural networks'. Raf. J. of Comp & Math. Vol.(11), No.(1). DOI: https://doi.org/10.33899/csmj.2014.163730

Battiti R. (1992). '1st and 2nd order method for learning betweensteepest descent and Newton method'. Neural Comp. 4(2). DOI: https://doi.org/10.1162/neco.1992.4.2.141

Fletcher R. and Reeves C. (1964).' function minimization byconjugate gradients'. Computer Journal. (7). DOI: https://doi.org/10.1093/comjnl/7.2.149

Noccedal J.(1996).' conjugate gradient methods and nonlinearoptimization. In [Adams.J.L.Nazareth (EDS.) linear and nonlinear conjugate gradient related methods. SIAM.

Parker D.(1987).'Optimal algorithms for adaptive Networks:second order back- propagation, second order Direct propagation,and second order Hebian learning.' Proceedings of the 1987 IEEE International Conference on Neural Networks, vol (2).

Andreas A. and Wu S. (2007).'Practical Optimization Algorithms and Engineering Applications', Springer Science and Business Media, LLC New York.

Livieris I., and pintelas P.(2011).'An advanced conjugate gradientraining algorithm based on a modified secant equation'. TechnicalReport No. TR11-03'. University of patras. Dep.of Math. GR-265 04 patras , Greece.

Andrei N.(2013). 'A simple three-term conjugate gradient f unconstrained optimization'. J. of Computation and AppliedMathematics (241). DOI: https://doi.org/10.1016/j.cam.2012.10.002

Zoutendijk G. (1970). 'Non-linear programming computational Methods', in J. Abadi (Ed). Integer and Non-linear Programming,North Holland.

Nguyen D. and Widrow B. (1990).' Improving the learning speedof 2-layer neural network by choosing initial values of adaptiveweights. Biological Cybernetics, 59. DOI: https://doi.org/10.1109/IJCNN.1990.137819

Shanno D. and Phua K.(1976).' Minimization of unconstrained multivariate functions'. ACM Transactions on Mathematical Software, 2. DOI: https://doi.org/10.1145/355666.355673

Downloads

Published

2015-06-30

How to Cite

Taqi, A. H. (2015). Improved Three-term Conjugate Gradient Algorithm For Training Neural Network. Journal of Kufa for Mathematics and Computer, 2(3), 93–100. https://doi.org/10.31642/JoKMC/2018/020309

Similar Articles

1 2 3 4 5 6 > >> 

You may also start an advanced similarity search for this article.