Advanced Sentiment Analysis Using Deep Learning: A Comprehensive Framework for High-Accuracy and Interpretable Models
Abstract
Sentiment analysis has become a critical tool for understanding public opinion, customer feedback, and social media trends. Despite significant advancements in deep learning, existing models often struggle with accuracy, generalizability, and interpretability, particularly when applied to complex and noisy datasets. In this paper, we propose a novel deep learning framework for sentiment analysis that addresses these limitations by combining the strengths of convolutional neural networks (CNNs) and transformer-based architectures. Our framework leverages verified and high-quality datasets, including Twitter Sentiment140, IMDb movie reviews, and Amazon product reviews, to ensure robustness and reliability. We introduce a hybrid model that integrates multi-head attention mechanisms with hierarchical feature extraction, enabling the model to capture both local and global contextual information effectively. Additionally, we employ state-of-the-art interpretability techniques, such as SHAP and LIME, to provide transparent and human-understandable explanations for model predictions. Experimental results demonstrate that our framework achieves superior performance compared to existing state-of-the-art methods, with an accuracy of 94.3%, an F1-score of 93.8%, and an AUC-ROC score of 97.2%. Furthermore, our model's interpretability features offer valuable insights into decision-making processes, making it highly applicable for real-world applications such as brand monitoring, market analysis, and political sentiment tracking. This study not only advances the field of sentiment analysis but also provides a scalable and interpretable solution for future research in natural language processing.
Downloads
References
[2] S. Minaee, N. Kalchbrenner, E. Cambria, N. Nikzad, M. Chenaghlu, and J. Gao, “Deep Learning--based Text Classification: A Comprehensive Review,” ACM Comput. Surv., vol. 54, no. 3, Apr. 2021, doi: 10.1145/3439726.
[3] Y. Zhang, R. Jin, and Z.-H. Zhou, “Understanding bag-of-words model: a statistical framework,” Int. J. Mach. Learn. Cybern., vol. 1, no. 1, pp. 43–52, 2010, doi: 10.1007/s13042-010-0001-0.
[4] K. Naithani and Y. P. Raiwani, “Sentiment Analysis on Social Media Data: A Survey,” in Innovations in Computer Science and Engineering, H. S. Saini, R. Sayal, A. Govardhan, and R. Buyya, Eds., Singapore: Springer Nature Singapore, 2023, pp. 735–745.
[5] D. Baishya and R. Baruah, “Recent Trends in Deep Learning for Natural Language Processing and Scope for Asian Languages,” in 2022 International Conference on Augmented Intelligence and Sustainable Systems (ICAISS), Nov. 2022, pp. 408–411. doi: 10.1109/ICAISS55157.2022.10010807.
[6] A. Vaswani et al., “Attention is all you need,” Adv. Neural Inf. Process. Syst., vol. 2017-December, no. Nips, pp. 5999–6009, 2017.
[7] A. Bhat and G. N. Jha, “Sarcasm Detection of Textual Data on Online SocialMedia: A Review,” in 2022 2nd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), Apr. 2022, pp. 1981–1985. doi: 10.1109/ICACITE53722.2022.9823869.
[8] M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, in KDD ’16. New York, NY, USA: Association for Computing Machinery, 2016, pp. 1135–1144. doi: 10.1145/2939672.2939778.
[9] J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” NAACL HLT 2019 - 2019 Conf. North Am. Chapter Assoc. Comput. Linguist. Hum. Lang. Technol. - Proc. Conf., vol. 1, no. Mlm, pp. 4171–4186, 2019.
[10] Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. Salakhutdinov, and Q. V Le, “XLNet: generalized autoregressive pretraining for language understanding,” in Proceedings of the 33rd International Conference on Neural Information Processing Systems, Red Hook, NY, USA: Curran Associates Inc., 2019.
[11] S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in Proceedings of the 31st International Conference on Neural Information Processing Systems, in NIPS’17. Red Hook, NY, USA: Curran Associates Inc., 2017, pp. 4768–4777.
[12] M. J. C. Samonte, A. T. G. Dela Rosa, L. J. C. Rivera, and J. S. E. Silo, “Using Hybrid CNN-LSTM Model for Sentiment Analysis of COVID-19 Tweets,” in 2023 13th International Conference on Software Technology and Engineering (ICSTE), Oct. 2023, pp. 133–142. doi: 10.1109/ICSTE61649.2023.00029.
[13] X. Wang, P. Liu, Z. Zhu, and R. Lu, “Aspect-based Sentiment Analysis with Graph Convolutional Networks over Dependency Awareness,” in 2022 26th International Conference on Pattern Recognition (ICPR), Aug. 2022, pp. 2238–2245. doi: 10.1109/ICPR56361.2022.9956479.
[14] H. Shao, “Research on sentiment analysis of weibo based on Improving Capsule Network,” in 2023 3rd International Conference on Consumer Electronics and Computer Engineering (ICCECE), Jan. 2023, pp. 620–623. doi: 10.1109/ICCECE58074.2023.10135216.
[15] Q. T. Nguyen, T. L. Nguyen, N. H. Luong, and Q. H. Ngo, “Fine-Tuning BERT for Sentiment Analysis of Vietnamese Reviews,” in 2020 7th NAFOSTED Conference on Information and Computer Science (NICS), Nov. 2020, pp. 302–307. doi: 10.1109/NICS51282.2020.9335899.
[16] V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter,” pp. 2–6, 2019, [Online]. Available: http://arxiv.org/abs/1910.01108
[17] Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, “Albert: a Lite Bert for Self-Supervised Learning of Language Representations,” 8th Int. Conf. Learn. Represent. ICLR 2020, pp. 1–17, 2020.
[18] J. Lee et al., “BioBERT: A pre-trained biomedical language representation model for biomedical text mining,” Bioinformatics, vol. 36, no. 4, pp. 1234–1240, 2020, doi: 10.1093/bioinformatics/btz682.
[19] Z. Liu, D. Huang, K. Huang, Z. Li, and J. Zhao, “FinBERT: a pre-trained financial language representation model for financial text mining,” in Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, in IJCAI’20. 2021.
[20] C. A. Córdova Sáenz and K. Becker, “Assessing the use of attention weights to interpret BERT-based stance classification,” in IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology, in WI-IAT ’21. New York, NY, USA: Association for Computing Machinery, 2022, pp. 194–201. doi: 10.1145/3486622.3493966.
[21] K. Hemker, Z. Shams, and M. Jamnik, “CGXplain: Rule-Based Deep Neural Network Explanations Using Dual Linear Programs,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 13932 LNCS, pp. 60–72, 2023, doi: 10.1007/978-3-031-39539-0_6.
[22] N. Liu and J. Zhao, “A BERT-Based Aspect-Level Sentiment Analysis Algorithm for Cross-Domain Text.,” Comput. Intell. Neurosci., vol. 2022, p. 8726621, 2022, doi: 10.1155/2022/8726621.
[23] M. Yekrangi and N. S. Nikolov, “Domain-Specific Sentiment Analysis: An Optimized Deep Learning Approach for the Financial Markets,” IEEE Access, vol. 11, pp. 70248–70262, 2023, doi: 10.1109/ACCESS.2023.3293733.
[24] A. Conneau et al., “Unsupervised cross-lingual representation learning at scale,” Proc. Annu. Meet. Assoc. Comput. Linguist., pp. 8440–8451, 2020, doi: 10.18653/v1/2020.acl-main.747.
[25] M. M. Agüero-Torales, J. I. Abreu Salas, and A. G. López-Herrera, “Deep learning and multilingual sentiment analysis on social media data: An overview,” Appl. Soft Comput., vol. 107, p. 107373, 2021, doi: https://doi.org/10.1016/j.asoc.2021.107373.
