OPTIMIZING SENTIMENT ANALYSIS WITH BERT ENHANCED BY BILSTM AND BIGRU LAYERS

Uma V; Ganesh V

doi:10.30572/2018/KJE/170206

Authors

Uma V Department of Computer and Information Science, Annamalai University, Cuddalore 608002, Tamil Nadu, India
Ganesh V Department of Computer Science, Government Arts College Autonomous, Kumbakonam 612002, Tamil Nadu, India

DOI:

https://doi.org/10.30572/2018/KJE/170206

Keywords:

Sentimental analysis, Machine learning, LSTM, Attention mechanism

Abstract

As mobile technology develops rapidly, social media becomes a rich source of platform for people to voice for his or her opinions and point of views. In order to support business decision making and policy making, we need to quantify public moods, sentiments by conducting sentiment analysis which emerges as a key research area in recent year. Many research efforts have been put into performing sentiment analysis, many tools and algorithms are devised to determine the sentiment as positive, negative or neutral, in social media. We improve the sentiment classfier by training a classical machine learning model on three different data sets. Although BERT has shown good performance in sentiment analysis, we need to find a way to raise the accuracy. We propose four Deep Learning models by combining BERT with Bidirectional Long ShortTerm Memory (BiLSTM) and Bidirectional Gated Recurrent Unit (BiGRU) algorithms. Our paper uses pre, trained word embedding vectors as inputs and aims to increase the accuracy of sentiment analysis and test the effect of BERT‘s combination with BiGRU and BiLSTM layers, DistilBERT and RoBERTa. We test the classification of text sentiments with and without use of emoji symbols. A comparative analysis of 2 pre, trained BERT models and 7 classical machine learning models suggests that the models with BiGRU layers achieve best performance in our sentiment analysis pipelines

Downloads

Download data is not yet available.

References

Abdul-Mageed M, Ungar L. Emonet: Fine-grained emotion detection with gated recurrent neural networks. In:Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: Long papers).2017.

Adoma AF, Comparative analyses of bert, roberta, distilbert, and xlnet for text-based emotion recognition. 17th International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), 2020.

Almatrafi O, Parack S, Chavan B. Application of location-based sentiment analysis using Twitter for identifying trends towards Indian general elections 2014. In: Proceedings of the 9th international conference on ubiquitous information management and communication. 2015.

Bansal B, Srivastava S. Hybrid attribute based sentiment classification of online reviews for consumer intelligence. Appl Intell. 2019;49(1):137–49.

Dang, N. C., Moreno-García, M. N., & De la Prieta, F. (2020). Sentiment analysis based on deep learning: a comparative study. Electronics, 9(3), 483.

Das A, Gambäck B. Sentimantics: conceptual spaces for lexical sentiment polarity representation with contextuality. In: Proceedings of the 3rd Workshop in Computational Approaches to Subjectivity and Sentiment Analysis. 2012.

Demotte, P., Wijegunarathna, K., Meedeniya, D. and Perera, I., 2023. Enhanced sentiment extraction architecture for social media content analysis using capsule networks. Multimedia tools and applications, 82(6), pp.8665-8690.

Giatsoglou M, et al. Sentiment analysis leveraging emotions and word embeddings. Expert Syst Appl. 2017;69:214–24.

Hashim, A. A., & Mazinani, M. (2025). Detection of keratoconus disease depending on corneal topography using deep learning. Kufa Journal of Engineering, 16(1). https://doi.org/10.30572/2018/KJE/160125

He Y. A Bayesian modeling approach to multi-dimensional sentiment distributions prediction. In: Proceedings of the First International Workshop on Issues of Sentiment Discovery and Opinion Mining. 2012.

Indrayuni, E., & Nurhadi, A. (2020). Optimizing genetic algorithms for sentiment analysis of apple product reviews using SVM. SinkrOn, 4(2), 172–178.

Janjua SH, et al. Multi-level aspect based sentiment classification of Twitter data: using hybrid approach in deep learning. PeerJ Comp Sci. 2021;7: e433.

Janjua, S. H., et al. (2021). Multi-level aspect-based sentiment classification of Twitter data: using a hybrid approach in deep learning. PeerJ Computer Science, 7, e433.

Karyotis C, et al. A fuzzy computational model of emotion for cloud based sentiment analysis. Inf Sci. 2018;433:448–63.

Kumawat, S., et al. (2021). Sentiment analysis using language models: a study. In: 2021 11th International Conference on Cloud Computing, Data Science & Engineering (Confluence). IEEE.

Ma Y, et al. Sentic LSTM: a hybrid network for targeted aspect-based sentiment analysis. Cogn Comput. 2018;10(4):639–50.

Maas A, et al. Learning word vectors for sentiment analysis. In: Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies. 2011.

Mohammed, H. A., Kareem, S. W., & Mohammed, A. S. (2022). A comparative evaluation of deep learning methods in digital image classification. Kufa Journal of Engineering, 13(4), Article 130405. https://doi.org/10.30572/2018/KJE/130405

Nimmi K, et al. Pre-trained ensemble model for identification of emotion during COVID-19 based on emergency response support system dataset. Appl Soft Comput. 2022;122: 108842.

Njølstad PCS, et al. Evaluating feature sets and classifiers for sentiment analysis of financial news. In: 2014 IEEE/WIC/ ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT). IEEE; 2014.

Pang B, Lee L, Vaithyanathan S, Thumbs up? Sentiment classification using machine learning techniques. arXiv preprint cs/0205070, 2002.

Prottasha NJ, Sami AA, Kowsher M, Murad SA, Bairagi AK, Masud M, Baz M. Transfer learning for sentiment analysis using BERT based supervised fine-tuning. Sensors. 2022;22:4157.

Sirisha U, Bolem SC. Aspect based sentiment & emotion analysis with ROBERTa, LSTM. IJACSA. 2022. https:// doi. org/ 10. 14569/ IJACSA. 2022. 01311 89.

Tan, K. L., et al. (2022). RoBERTa-LSTM: A hybrid model for sentiment analysis with transformer and recurrent neural network. IEEE Access, 10, 21517–21525.

Thapa B. Sentiment analysis of cybersecurity content on twitter and reddit. arXiv preprint arXiv: 2204. 12267, 2022.

Wang W, Xu H, Wan W. Implicit feature identification via hybrid association rule mining. Expert Syst Appl. 2013;40(9):3518–31.

Wen, S., & Li, J. (2018). Recurrent convolutional neural network with attention for Twitter and Yelp sentiment classification: ARC model for sentiment classification. In: Proceedings of the 2018 International Conference on Algorithms, Computing, and Artificial Intelligence.

Xia R, Zong C, Li S. Ensemble of feature sets and classification algorithms for sentiment classification. Inf Sci. 2011;181(6):1138–52.

Xiang, R., et al. (2020). Affection driven neural networks for sentiment analysis. In: Proceedings of the 12th Language Resources and Evaluation Conference. European Language Resources Association.

Zainuddin N, Selamat A, Ibrahim R. Hybrid sentiment classification on twitter aspect-based sentiment analysis. Appl Intell. 2018;48(5):1218–32.