TERNARY NEURAL NETWORK WITH SPARSE WEIGHT OPTIMIZATION FOR ENERGY-EFFICIENT MEMRISTOR CROSSBAR DEPLOYMENT WITH DEFECT TOLERANCE

Tien Nguyen Van

doi:10.30572/2018/KJE/170320

Authors

Tien Van Nguyen Institute of Engineering - Technology, Thu Dau Mot University, Ho Chi Minh City, Viet Nam

DOI:

https://doi.org/10.30572/2018/KJE/170320

Keywords:

Defects, Edge Devices, Memristor Crossbar, Neural Network, Sparsity

Abstract

Ternary neural network with three values of the weight (-1, 0, +1) is an effective solution for artificial intelligence applications in edge computing due to their energy efficiency, which is achieved through their deployment on memristor crossbars. However, the implementation on a memristor crossbar often has problems related to programming energy and defects. Therefore, this paper proposes a training method that uses both weight pruning and defect-aware training to optimize the number of programming pulses while maintaining a high recognition rate for the network. By using this method, at a 20% defect rate and 80% sparsity, the network achieves a 95% recognition rate on MNIST and 90% on CIFAR-10, compared to only 73.7% and 60.4% without using defect-aware training. At the same sparsity, the number of programming pulses, which is indicative of programming energy, is reduced by 49.1% compared to 20% sparsity

Downloads

Download data is not yet available.

References

Adam, G.C. et al. (2017) '3-D Memristor Crossbars for Analog and Neuromorphic Computing Applications', IEEE Transactions on Electron Devices, 64(1), pp. 312–318. DOI: https://doi.org/10.1109/TED.2016.2630925

Aidi Sharif, M. et al. (2025) 'Artificial Intelligence in Robotic Manipulators: Exploring Object Detection and Grasping Innovations', Kufa Journal of Engineering, 16(1), pp. 136–159. DOI: https://doi.org/10.30572/2018/KJE/160109

Alaa, R. et al. (2024) 'Object detection algorithms implementation on embedded devices: challenges and suggested solutions', Kufa Journal of Engineering, 15(3), pp. 148–169. DOI: https://doi.org/10.30572/2018/KJE/150309

Alemdar, H. et al. (2017) 'Ternary neural networks for resource-efficient AI applications', in International Joint Conference on Neural Networks (IJCNN), USA, pp. 2547–2554. DOI: https://doi.org/10.1109/IJCNN.2017.7966166

Bohr, M.T. and Young, I. (2017) 'CMOS Scaling Trends and Beyond', IEEE Micro, 37(6), pp. 20–29. DOI: https://doi.org/10.1109/MM.2017.4241347

Chakrabarti, B. et al. (2017) 'A multiply-add engine with monolithically integrated 3D memristor crossbar/CMOS hybrid circuit', Scientific Reports, 7, pp. 1–10. DOI: https://doi.org/10.1038/srep42429

Deng, L. (2012) 'The MNIST Database of Handwritten Digit Images for Machine Learning Research', IEEE Signal Processing Magazine, 29(6), pp. 141–142. DOI: https://doi.org/10.1109/MSP.2012.2211477

Graves, C.E. et al. (2020) 'In-Memory Computing with Memristor Content Addressable Memories for Pattern Matching', Advanced Materials, 32(38), 2003437. DOI: https://doi.org/10.1002/adma.202003437

He, K. et al. (2016) 'Deep residual learning for image recognition', in Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), USA, pp. 770–778. DOI: https://doi.org/10.1109/CVPR.2016.90

Hu, M. et al. (2018) 'Memristor-based analog computation and neural network classification with a dot product engine', Advanced Materials, 30(9), 1705914. DOI: https://doi.org/10.1002/adma.201705914

Indiveri, G. and Liu, S.-C. (2015) 'Memory and Information Processing in Neuromorphic Systems', Proceedings of the IEEE, 103(8), pp. 1379–1397. DOI: https://doi.org/10.1109/JPROC.2015.2444094

Jang, J.T. et al. (2018) 'Effect of oxygen content of the LaAlO3 layer on the synaptic behavior of Pt/LaAlO3/Nb-doped SrTiO3 memristors for neuromorphic applications', Solid-State Electronics, 140, pp. 139–143. DOI: https://doi.org/10.1016/j.sse.2017.10.032

Jo, S.H. et al. (2010) 'Nanoscale memristor device as synapse in neuromorphic systems', Nano Letters, 10(4), pp. 1297–1301. DOI: https://doi.org/10.1021/nl904092h

Krizhevsky, A., Nair, V. and Hinton, G. (2018) CIFAR-10 and CIFAR-100 Datasets. https://www.cs.toronto.edu/~kriz/cifar.html [accessed on 8 August 2025]

Linn, E. et al. (2012) 'Beyond von Neumann - Logic operations in passive crossbar arrays alongside memory operations', Nanotechnology, 23(30), 305205. DOI: https://doi.org/10.1088/0957-4484/23/30/305205

Lin, P. et al. (2020) 'Three-dimensional memristor circuits as complex neural networks', Nature Electronics, 3(5), pp. 225–232. DOI: https://doi.org/10.1038/s41928-020-0397-9

Mohammed, H. et al. (2022) 'A comparative evaluation of deep learning methods in digital image classification', Kufa Journal of Engineering, 13(4), pp. 53–69. DOI: https://doi.org/10.30572/2018/KJE/130405

Nguyen, T.V., An, J. and Min, K.-S. (2021) 'Memristor-CMOS hybrid neuron circuit with nonideal-effect correction related to parasitic resistance for binary-memristor-crossbar neural networks', Micromachines, 12(7), 791. DOI: https://doi.org/10.3390/mi12070791

Nguyen, T.V., Pham, K.V. and Min, K.-S. (2019) 'Hybrid circuit of memristor and complementary metal–oxide–semiconductor for defect-tolerant spatial pooling with Boost-factor Adjustment', Materials, 12(13), 2122. DOI: https://doi.org/10.3390/ma12132122

Pham, K.V. et al. (2018) 'Memristor binarized neural networks', Journal of Semiconductor Technology and Science, 18(5), pp. 568–577. DOI: https://doi.org/10.5573/JSTS.2018.18.5.568

Pham, K.V., Tran, S.B., Nguyen, T.V. and Min, K.-S. (2019) 'Asymmetrical training scheme of binary-memristor-crossbar-based neural networks for energy-efficient edge-computing nanoscale systems', Micromachines, 10(3), 141. DOI: https://doi.org/10.3390/mi10020141

Pouyanfar, S. et al. (2018) 'A survey on deep learning: Algorithms, techniques, and applications', ACM Computing Surveys, 51(1), pp. 1–36. DOI: https://doi.org/10.1145/3234150

Sebastian, A., Le Gallo, M. and Eleftheriou, E. (2019) 'Computational phase-change memory: Beyond von Neumann computing', Journal of Physics D: Applied Physics, 52(44), 443002. DOI: https://doi.org/10.1088/1361-6463/ab37b6

Sheng, X. et al. (2019) 'Low-Conductance and Multilevel CMOS-Integrated Nanoscale Oxide Memristors', Advanced Electronic Materials, 5(3), 1800876. DOI: https://doi.org/10.1002/aelm.201800876

Shrestha, A. and Mahmood, A. (2019) 'Review of Deep Learning Algorithms and Architectures', IEEE Access, 7, pp. 53040–53065. DOI: https://doi.org/10.1109/ACCESS.2019.2912200

Song, C. et al. (2017) 'A quantization-aware regularized learning method in multilevel memristor-based neuromorphic computing system', in IEEE 6th Non-Volatile Memory Systems and Applications Symposium (NVMSA), Hsinchu, Taiwan, 16–18 August, pp. 1–6. DOI: https://doi.org/10.1109/NVMSA.2017.8064465

Strukov, D.B. et al. (2008) 'The missing memristor found', Nature, 453(7191), pp. 80–83. DOI: https://doi.org/10.1038/nature06932

Truong, S.N. et al. (2016) 'New pulse amplitude modulation for fine tuning of memristor synapses', Microelectronics Journal, 55, pp. 162–168. DOI: https://doi.org/10.1016/j.mejo.2016.07.010

Wang, T. et al. (2020) 'Three-Dimensional Nanoscale Flexible Memristor Networks with Ultralow Power for Information Transmission and Processing Application', Nano Letters, 20(6), pp. 4111–4120. DOI: https://doi.org/10.1021/acs.nanolett.9b05271

Wright, C.D. et al. (2013) 'Beyond von-Neumann computing with nanoscale phase-change memory devices', Advanced Functional Materials, 23(18), pp. 2248–2254. DOI: https://doi.org/10.1002/adfm.201202383

Yeo, I. et al. (2019) 'Stuck-at-fault tolerant schemes for memristor crossbar array-based neural networks', IEEE Transactions on Electron Devices, 66(7), pp. 2937–2945. DOI: https://doi.org/10.1109/TED.2019.2914460