An Approach for Emotion Detection Using a Hybrid Vision Sequence-Based Convolutional Neural Networks (Vi-CNN) Model
DOI:
https://doi.org/10.31642/JoKMC/2018/120205Keywords:
Deep learning, emotion recognition, Vision transformer, Convolutional Neural Network, text data.Abstract
Emotion represents the significant reflection or indicator of human mental states, biological effects, and physiological statements, and it also plays a vital role in human interpersonal communication and decision-making. People convey their emotions naturally via everyday interactive communication due to the increasing advancement of social media platforms. Nowadays, detecting emotions from extensive textual data helps to provide expressive information for understanding the behavior of humans. Meanwhile, most prior techniques used for emotion detection are insufficient in providing promising results from long-term contextual information. Thus, it motivates introducing a hybrid deep learning model, Vision sequence-based Convolutional Neural Network (Vi-CNN), from text data to detect emotion. The Bidirectional Encoder Representation from Transformer (BERT) transforms the input texts into tokens. Then, extracting the most suitable and appropriate features from the text tokens is executed. Following this, the Vi-CNN model significantly detects emotions from the extracted features and classifies the recognized emotions. Furthermore, the results obtained by Vi-CNN are compared with those of other prevailing schemes. The experimental results highlight that the Vi-CNN attained promising results with maximum recall of 94.765%, precision of 92.988%, and F-measure of 93.867%.
Downloads
References
[1] Saxena, A., Khanna, A. and Gupta, D., “Emotion recognition and detection methods: A comprehensive survey”, Journal of Artificial Intelligence and Systems, vol.2, no.1, pp.53-79, 2020 doi: https://doi.org/10.33969/AIS.2020.21005. DOI: https://doi.org/10.33969/AIS.2020.21005
[2] Khare, S.K., Blanes-Vidal, V., Nadimi, E.S. and Acharya, U.R., “Emotion recognition and artificial intelligence: A systematic review (2014–2023) and research recommendations”, Information Fusion, vol.102, pp.102019, 2024. doi:https://doi.org/10.1016/j.inffus.2023.102019. DOI: https://doi.org/10.1016/j.inffus.2023.102019
[3] Torres-Valencia, C., Álvarez-López, M. and Orozco-Gutiérrez, Á., “SVM-based feature selection methods for emotion recognition from multimodal data”, Journal on Multimodal User Interfaces, vol.11, pp.9-23, 2017. https://doi.org/10.1007/s12193-016-0222-y DOI: https://doi.org/10.1007/s12193-016-0222-y
[4] Abas, A.R., Elhenawy, I., Zidan, M. and Othman, M., “BERT-CNN: A Deep Learning Model for Detecting Emotions from Text”, Computers, Materials & Continua, vol.71, no.2, 2022. https://doi.org/10.32604/cmc.2022.021671 DOI: https://doi.org/10.32604/cmc.2022.021671
[5] Halim, Z., Waqar, M. and Tahir, M., “A machine learning-based investigation utilizing the in-text features for the identification of dominant emotion in an email”, Knowledge-based systems, vol.208, pp.106443, 2020. https://doi.org/10.1016/j.knosys.2020.106443 DOI: https://doi.org/10.1016/j.knosys.2020.106443
[6] Doewes, R.I., Gangadhar, L. and Subburaj, S., “An overview on stress neurobiology: Fundamental concepts and its consequences”, Neuroscience Informatics, vol.1, no.3, pp.100011, 2021, doi:https://doi.org/10.1016/j.neuri.2021.100011 DOI: https://doi.org/10.1016/j.neuri.2021.100011
[7] Shelke, N., Chaudhury, S., Chakrabarti, S., Bangare, S.L., Yogapriya, G. and Pandey, P., “An efficient way of text-based emotion analysis from social media using LRA-DNN”, Neuroscience Informatics, vol.2, no.3, pp.100048, 2022, doi:https://doi.org/10.1016/j.neuri.2022.100048 DOI: https://doi.org/10.1016/j.neuri.2022.100048
[8] Wen, J., Jiang, D., Tu, G., Liu, C. and Cambria, E., “Dynamic interactive multiview memory network for emotion recognition in conversation”, Information Fusion, vol.91, pp.123-133, 2023, doi:https://doi.org/10.1016/j.inffus.2022.10.009 DOI: https://doi.org/10.1016/j.inffus.2022.10.009
[9] Adikari, A., Gamage, G., De Silva, D., Mills, N., Wong, S.M.J. and Alahakoon, D., “A self structuring artificial intelligence framework for deep emotions modeling and analysis on the social web”, Future Generation Computer Systems, vol.116, pp.302-315, 2021, doi:https://doi.org/10.1016/j.future.2020.10.028 DOI: https://doi.org/10.1016/j.future.2020.10.028
[10] Jianqiang, Z., Xiaolin, G. and Xuejun, Z., “Deep convolution neural networks for twitter sentiment analysis”, IEEE access, vol.6, pp.23253-23260, 2018. doi: https://doi.org/10.35940/ijitee.i1107.0789s419. DOI: https://doi.org/10.1109/ACCESS.2017.2776930
[11] Mohammad, F., Khan, M., Marwat, S.N.K., Jan, N., Gohar, N., Bilal, M. and Al-Rasheed, A., “Text augmentation-based model for emotion recognition using transformers”, CMC-COMPUTERS MATERIALS & CONTINUA, vol.76, no.3, pp.3523-3547, 2023, doi: https://doi.org/10.32604/cmc.2023.040202. DOI: https://doi.org/10.32604/cmc.2023.040202
[12] Batbaatar, E., Li, M. and Ryu, K.H., “Semantic-emotion neural network for emotion recognition from text”, IEEE access, vol.7, pp.111866-111878, 2019. doi: https://doi.org/10.1109/ACCESS.2019.2934529. DOI: https://doi.org/10.1109/ACCESS.2019.2934529
[13] Han, T., Zhang, Z., Ren, M., Dong, C., Jiang, X. and Zhuang, Q., “Text emotion recognition based on XLNet-BiGRU-Att”, Electronics, vol.12, no.12, pp.2704, 2023, doi: https://doi.org/10.3390/electronics12122704 DOI: https://doi.org/10.3390/electronics12122704
[14] Sailunaz, K. and Alhajj, R., “Emotion and sentiment analysis from Twitter text”, Journal of computational science, vol.36, pp.101003, 2019, doi: https://doi.org/10.1016/j.jocs.2019.05.009 DOI: https://doi.org/10.1016/j.jocs.2019.05.009
[15] Mohammad, S.M. and Bravo-Marquez, F., “WASSA-2017 shared task on emotion intensity”, arXiv preprint arXiv:1708.03700, 2017, doi:https://doi.org/10.48550/arXiv.1708.03700 DOI: https://doi.org/10.18653/v1/W17-5205
[16] Bharti, S.K., Varadhaganapathy, S., Gupta, R.K., Shukla, P.K., Bouye, M., Hingaa, S.K. and Mahmoud, A., “Text‐Based Emotion Recognition Using Deep Learning Approach”, Computational Intelligence and Neuroscience, vol.2022, no.1, pp.2645381, 2022, doi: https://doi.org/10.1155/2022/2645381 DOI: https://doi.org/10.1155/2022/2645381
[17] Rastogi, G. and Sushil, R., “Cloud computing implementation: key issues and solution”, In proceedings of 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom, pp. 320-324, March, 2015.
[18] Irfan, S., Anjum, N., Althobaiti, T., Alotaibi, A.A., Siddiqui, A.B. and Ramzan, N., “Heartbeat classification and arrhythmia detection using a multi-model deep-learning technique”, Sensors, vol.22, no.15, pp.5606, 2022, doi:https://doi.org/10.3390/s22155606 DOI: https://doi.org/10.3390/s22155606
[19] Wani, T.M., Gunawan, T.S., Qadri, S.A.A., Kartiwi, M. and Ambikairajah, E., “A comprehensive review of speech emotion recognition systems”, IEEE access, vol.9, pp.47795-47814, 2021, doi: https://doi.org/10.1109/ACCESS.2021.3068045 DOI: https://doi.org/10.1109/ACCESS.2021.3068045
[20] Tu, G., Wen, J., Liu, C., Jiang, D. and Cambria, E., “Context-and sentiment-aware networks for emotion recognition in conversation”, IEEE Transactions on Artificial Intelligence, vol.3, no.5, pp.699-708, 2022, doi: https://doi.org/10.1109/TAI.2022.3149234 DOI: https://doi.org/10.1109/TAI.2022.3149234
[21] Asghar, M.Z., Lajis, A., Alam, M.M., Rahmat, M.K., Nasir, H.M., Ahmad, H., Al-Rakhami, M.S., Al-Amri, A. and Albogamy, F.R., “A deep neural network model for the detection and classification of emotions from textual content”, Complexity, vol.2022, no.1, pp.8221121, 2022, doi: https://doi.org/10.1155/2022/8221121 DOI: https://doi.org/10.1155/2022/8221121
[22] Ghafoor, Y., Jinping, S., Calderon, F.H., Huang, Y.H., Chen, K.T. and Chen, Y.S., “TERMS: textual emotion recognition in multidimensional space”, Applied Intelligence, vol.53, no.3, pp.2673-2693, 2023, doi:https://doi.org/10.1007/s10489-022-03567-4 DOI: https://doi.org/10.1007/s10489-022-03567-4
[23] Emotion detection from text dataset is taken from “https://www.kaggle.com/datasets/pashupatigupta/emotion-detection-from-text?resource=download” accessed on November 2024.
[24] Grail, Q., Perez, J. and Gaussier, E., “Globalizing BERT-based transformer architectures for long document summarization”, In Proceedings of the 16th conference of the European chapter of the Association for Computational Linguistics: Main volume, pp. 1792-1810, 2021, doi:https://doi.org/10.18653/v1/2021.eacl-main.154 DOI: https://doi.org/10.18653/v1/2021.eacl-main.154
[25] Sundaram, V., Ahmed, S., Muqtadeer, S.A. and Reddy, R.R., “Emotion analysis in text using TF-IDF”, In proceedings of 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence), pp. 292-297, January, 2021, doi:https://doi.org/10.1109/Confluence51648.2021.9377159 DOI: https://doi.org/10.1109/Confluence51648.2021.9377159
[26] Thakur, R.K. and Deshpande, M.V., “Kernel Optimized-Support Vector Machine and Mapreduce framework for sentiment classification of train reviews”, International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol.27, no.06, pp.1025-1050, 2019, doi:https://doi.org/10.1142/S0218488519500454 DOI: https://doi.org/10.1142/S0218488519500454
[27] Gupta, A. and Goyal, K.K., “Classification of Semantic Similarity Technique between Word Pairs using Word Net”, International Journal of Engineering and Advanced Technology, vol.9, No.2, pp.4397-4402, 2019. DOI: https://doi.org/10.35940/ijeat.B2961.129219
[28] Shrivastava, K., Kumar, S. and Jain, D.K., “An effective approach for emotion detection in multimedia text data using sequence based convolutional neural network”, Multimedia tools and applications, vol.78, pp.29607-29639, 2019, doi:https://doi.org/10.1007/s11042-019-07813-9 DOI: https://doi.org/10.1007/s11042-019-07813-9
[29] Yin, H., Vahdat, A., Alvarez, J.M., Mallya, A., Kautz, J. and Molchanov, P., “A-vit: Adaptive tokens for efficient vision transformer”, In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10809-10818, 2022, doi: https://doi.org/10.1109/cvpr52688.2022.01054. DOI: https://doi.org/10.1109/CVPR52688.2022.01054
[30] Suanmali, L., Salim, N. and Binwahlan, M.S., “Feature-based sentence extraction using fuzzy inference rules”, In Proceedings of 2009 International Conference on Signal Processing Systems, IEEE, pp. 511-515, 2009, doi:https://doi.org/10.1109/ICSPS.2009.156 DOI: https://doi.org/10.1109/ICSPS.2009.156
[31] Ghosh, M. and Kar, A., “Unsupervised linguistic approach for sentiment classification from online reviews using SentiWordNet 3.0”, Int J Eng Res Technol, vol.2, no.9, pp.1-6, 2013,
[32] Abubakar, H.D., Umar, M. and Bakale, M.A., “Sentiment classification: Review of text vectorization methods: Bag of words, Tf-Idf, Word2vec and Doc2vec"”, SLU Journal of Science and Technology, vol.4, no.1, pp.27-33, 2022, doi:https://doi.org/10.56471/slujst.v4i.266 DOI: https://doi.org/10.56471/slujst.v4i.266
[33] Damsleth, E. and Spjøtvoll, E., “Estimation of trigonometric components in time series”, Journal of the American Statistical Association, vol.77, no.378, pp.381-387, 1982, doi:https://doi.org/10.1080/01621459.1982.10477820 DOI: https://doi.org/10.1080/01621459.1982.10477820
[34] J. G. Proakis and D. G. Manolakis, Digital signal processing : principles, algorithms, and applications. Upper Saddle River, N.J.: Prentice Hall, 1996.
[35] A. V. Oppenheim and R. W. Schafer, Discrete-time signal processing. Upper Saddle River: Pearson, 2010.
[36] J. Opitz, “A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice,” Transactions of the Association for Computational Linguistics, vol. 12, pp. 820–836, Jan. 2024, doi: https://doi.org/10.1162/tacl_a_00675. DOI: https://doi.org/10.1162/tacl_a_00675
Downloads
Published
Issue
Section
Categories
License
Copyright (c) 2025 Vian Sabeeh, Ahmed Bahaaulddin Alwahhab
This work is licensed under a Creative Commons Attribution 4.0 International License.
which allows users to copy, create extracts, abstracts, and new works from the Article, alter and revise the Article, and make commercial use of the Article (including reuse and/or resale of the Article by commercial entities), provided the user gives appropriate credit (with a link to the formal publication through the relevant DOI), provides a link to the license, indicates if changes were made and the licensor is not represented as endorsing the use made of the work.
How to Cite
