Sentiment Analysis in the Age of Transformers and Large Language Models: A Comprehensive Review of Recent Advances and Future
Keywords:
Sentiment Analysis, Large Language Models, Transformers, Bias Mitigation, Explainable AI
Abstract
Sentiment analysis (SA) has undergone a significant transformation with the emergence of Transformer-based models and Large Language Models (LLMs). This review provides a comprehensive overview of recent advances in Transformer-based SA, highlighting the impact of LLMs on accuracy, nuance, and context-awareness. Architectural innovations, domain adaptation techniques, and methods for handling context and long-range dependencies are explored. The review also addresses the critical challenges and limitations associated with LLMs in SA, including bias and fairness, interpretability and explainability, data scarcity, computational cost, and robustness against adversarial attacks. Mitigation strategies and best practices are examined, focusing on data augmentation, adversarial training, bias-aware training objectives, attention visualization, and model distillation. Finally, the review outlines future research directions, emphasizing multimodal sentiment analysis, explainable AI, ethical considerations, low-resource languages, and domain-specific applications. This work concludes that while LLMs offer unprecedented opportunities for advancing SA, addressing the identified challenges is crucial for ensuring responsible and effective deployment in real-world scenarios.
Downloads
Download data is not yet available.
References
B. Liu, “Sentiment Analysis and Opinion Mining,” 2012, doi: 10.1007/978-3-031-02145-9.
[2] B. Pang and L. Lee, Opinion Mining and Sentiment Analysis. now, 2008. doi: 10.1561/1500000011.
[3] A. Vaswani et al., “Attention is all you need,” Adv. Neural Inf. Process. Syst., vol. 2017-Decem, no. Nips, pp. 5999–6009, 2017.
[4] J. Devlin, M.-W. Chang, K. Lee, K. T. Google, and A. I. Language, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Naacl-Hlt 2019, no. Mlm, pp. 4171–4186, 2018, [Online]. Available: https://aclanthology.org/N19-1423.pdf
[5] Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” no. 1, 2019, [Online]. Available: http://arxiv.org/abs/1907.11692
[6] T. B. Brown et al., “Language models are few-shot learners,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, in NIPS ’20. Red Hook, NY, USA: Curran Associates Inc., 2020.
[7] A. H. Musawa, Ricky, I. A. Iswanto, and M. F. Hidayat, “Exploring Transformer-Based Model in Sentiment Analysis of Movie Review,” in 2024 8th International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE), Aug. 2024, pp. 214–219. doi: 10.1109/ICITISEE63424.2024.10730492.
[8] X. Zhu, S. Gardiner, T. Roldán, and D. Rossouw, “The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in the Era of Large Language Models,” pp. 141–152, 2024, [Online]. Available: https://aclanthology.org/2024.wassa-1.12
[9] P. Xue and W. Bai, “A Fine-Grained Sentiment Analysis Method Using Transformer for Weibo Comment Text,” Int. J. Inf. Technol. Syst. Appoach, vol. 17, no. 1, pp. 1–24, Jul. 2024, doi: 10.4018/IJITSA.345397.
[10] T. Bolukbasi, K.-W. Chang, J. Zou, V. Saligrama, and A. Kalai, “Man is to computer programmer as woman is to homemaker? debiasing word embeddings,” in Proceedings of the 30th International Conference on Neural Information Processing Systems, in NIPS’16. Red Hook, NY, USA: Curran Associates Inc., 2016, pp. 4356–4364.
[11] M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, in KDD ’16. New York, NY, USA: Association for Computing Machinery, 2016, pp. 1135–1144. doi: 10.1145/2939672.2939778.
[12] M. Ferrari Dacrema, P. Cremonesi, and D. Jannach, “Are we really making much progress? A worrying analysis of recent neural recommendation approaches,” in Proceedings of the 13th ACM Conference on Recommender Systems, in RecSys ’19. New York, NY, USA: Association for Computing Machinery, 2019, pp. 101–109. doi: 10.1145/3298689.3347058.
[13] R. Stefan, G. Carutasu, and M. Mocan, “Ethical Considerations in the Implementation and Usage of Large Language Models,” in The 17th International Conference Interdisciplinarity in Engineering, L. Moldovan and A. Gligor, Eds., Cham: Springer Nature Switzerland, 2024, pp. 131–144.
[14] J. P. Venugopal, A. A. V. Subramanian, G. Sundaram, M. Rivera, and P. Wheeler, “A Comprehensive Approach to Bias Mitigation for Sentiment Analysis of Social Media Data,” Appl. Sci., vol. 14, no. 23, 2024, doi: 10.3390/app142311471.
[15] K. L. Revathi, A. R. Satish, and P. S. Rao, “Fine - Grained Sentiment Analysis on Online Reviews,” in 2023 Third International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT), Jan. 2023, pp. 1–6. doi: 10.1109/ICAECT57570.2023.10118291.
[16] H. Zheng, J. Zhang, Y. Suzuki, F. Fukumoto, and H. Nishizaki, “Semi-Supervised Learning for Aspect-Based Sentiment Analysis,” in 2021 International Conference on Cyberworlds (CW), 2021, pp. 209–212. doi: 10.1109/CW52790.2021.00042.
[17] A. D. L. Languré and M. Zareei, “Breaking Barriers in Sentiment Analysis and Text Emotion Detection: Toward a Unified Assessment Framework,” IEEE Access, vol. 11, pp. 125698–125715, 2023, doi: 10.1109/ACCESS.2023.3331323.
[18] S. H. Lye and P. L. Teh, “Customer Intent Prediction using Sentiment Analysis Techniques,” in 2021 11th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), 2021, pp. 185–190. doi: 10.1109/IDAACS53288.2021.9660391.
[19] J. Y. M. Nip and B. Berthelier, “Social Media Sentiment Analysis,” Encyclopedia, vol. 4, no. 4, pp. 1590–1598, 2024, doi: 10.3390/encyclopedia4040104.
[20] A. PATEL, P. OZA, and S. AGRAWAL, “Sentiment Analysis of Customer Feedback and Reviews for Airline Services using Language Representation Model,” Procedia Comput. Sci., vol. 218, pp. 2459–2467, 2023, doi: https://doi.org/10.1016/j.procs.2023.01.221.
[21] C. P. Gupta and V. V Ravi Kumar, “Sentiment Analysis and its Application in Analysing Consumer Behaviour,” in 2023 International Conference on Emerging Techniques in Computational Intelligence (ICETCI), 2023, pp. 332–337. doi: 10.1109/ICETCI58599.2023.10331537.
[22] M. Z. Ansari, M. B. Aziz, M. O. Siddiqui, H. Mehra, and K. P. Singh, “Analysis of Political Sentiment Orientations on Twitter,” Procedia Comput. Sci., vol. 167, pp. 1821–1828, 2020, doi: https://doi.org/10.1016/j.procs.2020.03.201.
[23] P. M. Madan, M. R. Madan, and D. P. Thakur, “Analysing The Patient Sentiments in Healthcare Domain Using Machine Learning,” Procedia Comput. Sci., vol. 238, pp. 683–690, 2024, doi: https://doi.org/10.1016/j.procs.2024.06.077.
[24] P. Thangavel and R. Lourdusamy, “A lexicon-based approach for sentiment analysis of multimodal content in tweets,” Multimed. Tools Appl., vol. 82, no. 16, pp. 24203–24226, 2023, doi: 10.1007/s11042-023-14411-3.
[25] A. Briciu, A.-D. Călin, D.-L. Miholca, C. Moroz-Dubenco, V. Petrașcu, and G. Dascălu, “Machine-Learning-Based Approaches for Multi-Level Sentiment Analysis of Romanian Reviews,” Mathematics, vol. 12, no. 3, 2024, doi: 10.3390/math12030456.
[26] R. Jozefowicz, W. Zaremba, and I. Sutskever, “An empirical exploration of recurrent network architectures,” in Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, in ICML’15. JMLR.org, 2015, pp. 2342–2350.
[27] I. Perikos and A. Diamantopoulos, “Explainable Aspect-Based Sentiment Analysis Using Transformer Models,” Big Data Cogn. Comput., vol. 8, no. 11, 2024, doi: 10.3390/bdcc8110141.
[28] M. Kasri, M. Birjali, M. Nabil, A. Beni-Hssane, A. El-Ansari, and M. El Fissaoui, “Refining Word Embeddings with Sentiment Information for Sentiment Analysis,” J. ICT Stand., vol. 10, no. 3, pp. 353–382, 2022, doi: 10.13052/jicts2245-800X.1031.
[29] C. Yue, A. Li, Z. Chen, G. Luan, and S. Guo, “Domain-Aware Neural Network with a Novel Attention-Pooling Technology for Binary Sentiment Classification,” Appl. Sci., vol. 14, no. 17, 2024, doi: 10.3390/app14177971.
[30] W. Li, D. Li, H. Yin, L. Zhang, Z. Zhu, and P. Liu, “Lexicon-Enhanced Attention Network Based on Text Representation for Sentiment Classification,” Appl. Sci., vol. 9, no. 18, 2019, doi: 10.3390/app9183717.
[31] M. Rizinski, H. Peshov, K. Mishev, M. Jovanovik, and D. Trajanov, “Sentiment Analysis in Finance: From Transformers Back to eXplainable Lexicons (XLex),” IEEE Access, vol. 12, pp. 7170–7198, 2024, doi: 10.1109/ACCESS.2024.3349970.
[32] M. Bilal and A. A. Almazroi, “Effectiveness of Fine-tuned BERT Model in Classification of Helpful and Unhelpful Online Customer Reviews,” Electron. Commer. Res., vol. 23, no. 4, pp. 2737–2757, 2023, doi: 10.1007/s10660-022-09560-w.
[33] Z. Li, L. Zhou, X. Yang, H. Jia, W. Li, and J. Zhang, “User Sentiment Analysis of COVID-19 via Adversarial Training Based on the BERT-FGM-BiGRU Model,” Systems, vol. 11, no. 3, 2023, doi: 10.3390/systems11030129.
[34] G. Zhao, Y. Luo, Q. Chen, and X. Qian, “Aspect-based sentiment analysis via multitask learning for online reviews,” Knowledge-Based Syst., vol. 264, p. 110326, 2023, doi: https://doi.org/10.1016/j.knosys.2023.110326.
[35] B. Liang et al., “Few-shot Aspect Category Sentiment Analysis via Meta-learning,” ACM Trans. Inf. Syst., vol. 41, no. 1, Jan. 2023, doi: 10.1145/3529954.
[36] P. Wang, J. Li, and J. Hou, “S2SAN: A sentence-to-sentence attention network for sentiment analysis of online reviews,” Decis. Support Syst., vol. 149, p. 113603, 2021, doi: https://doi.org/10.1016/j.dss.2021.113603.
[37] D. Lee, C. S. Prakash, J. FitzGerald, and J. Lehmann, “MATTER: Memory-Augmented Transformer Using Heterogeneous Knowledge Sources,” 2024, [Online]. Available: http://arxiv.org/abs/2406.04670
[38] B. Liu, W. Guan, C. Yang, Z. Fang, and Z. Lu, “Transformer and Graph Convolutional Network for Text Classification,” Int. J. Comput. Intell. Syst., vol. 16, no. 1, p. 161, 2023, doi: 10.1007/s44196-023-00337-z.
[39] C. Yeh, Y. Chen, A. Wu, C. Chen, F. Viegas, and M. Wattenberg, “AttentionViz: A Global View of Transformer Attention,” IEEE Trans. Vis. Comput. Graph., vol. 30, no. 1, pp. 262–272, 2024, doi: 10.1109/TVCG.2023.3327163.
[40] J.-S. Pan, G.-L. Wang, S.-C. Chu, D. Yang, and V. Snášel, “New feature attribution method for explainable aspect-based sentiment classification,” Knowledge-Based Syst., vol. 304, p. 112550, 2024, doi: https://doi.org/10.1016/j.knosys.2024.112550.
[41] S. Sivakumar and R. Rajalakshmi, “Context-aware sentiment analysis with attention-enhanced features from bidirectional transformers,” Soc. Netw. Anal. Min., vol. 12, no. 1, p. 104, 2022, doi: 10.1007/s13278-022-00910-y.
[42] A. Shirgaonkar, N. Pandey, N. C. Abay, T. Aktas, and V. Aski, “Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Data,” 2024, [Online]. Available: http://arxiv.org/abs/2410.18588
[43] D. You and D. Chon, “Trust & Safety of LLMs and LLMs in Trust & Safety,” 2024, [Online]. Available: http://arxiv.org/abs/2412.02113
[44] Y. Guo et al., “Bias in Large Language Models: Origin, Evaluation, and Mitigation,” pp. 1–47, 2024, [Online]. Available: http://arxiv.org/abs/2411.10915
[45] X. Fang, S. Che, M. Mao, H. Zhang, M. Zhao, and X. Zhao, “Bias of AI-generated content: an examination of news produced by large language models.,” Sci. Rep., vol. 14, no. 1, p. 5224, Mar. 2024, doi: 10.1038/s41598-024-55686-2.
[46] L. H. X. Ng, I. Cruickshank, and R. K.-W. Lee, “Examining the Influence of Political Bias on Large Language Model Performance in Stance Classification,” 2024, [Online]. Available: http://arxiv.org/abs/2407.17688
[47] G. Kuzmin, N. Yadav, I. Smirnov, T. Baldwin, and A. Shelmanov, “Inference-Time Selective Debiasing to Enhance Fairness in Text Classification Models,” 2024, [Online]. Available: http://arxiv.org/abs/2407.19345
[48] N. Raghunathan and K. Saravanakumar, “Challenges and Issues in Sentiment Analysis: A Comprehensive Survey,” IEEE Access, vol. 11, no. June, pp. 69626–69642, 2023, doi: 10.1109/ACCESS.2023.3293041.
[49] F. Ronchini, L. Comanducci, and F. Antonacci, “Synthetic training set generation using text-to-audio models for environmental sound classification,” 2024, [Online]. Available: http://arxiv.org/abs/2403.17864
[50] D. Sanmartín and V. Prohaska, “Exploring Tpus for Ai Applications,” 2023.
[51] N. Wang, K. Walter, Y. Gao, and A. Abuadbba, “Through the Lens of Attack Objectives,” pp. 1–15.
[52] H. Waghela, “Robust Image Classification : Defensive Strategies against FGSM and PGD Adversarial Attacks”.
[2] B. Pang and L. Lee, Opinion Mining and Sentiment Analysis. now, 2008. doi: 10.1561/1500000011.
[3] A. Vaswani et al., “Attention is all you need,” Adv. Neural Inf. Process. Syst., vol. 2017-Decem, no. Nips, pp. 5999–6009, 2017.
[4] J. Devlin, M.-W. Chang, K. Lee, K. T. Google, and A. I. Language, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” Naacl-Hlt 2019, no. Mlm, pp. 4171–4186, 2018, [Online]. Available: https://aclanthology.org/N19-1423.pdf
[5] Y. Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” no. 1, 2019, [Online]. Available: http://arxiv.org/abs/1907.11692
[6] T. B. Brown et al., “Language models are few-shot learners,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, in NIPS ’20. Red Hook, NY, USA: Curran Associates Inc., 2020.
[7] A. H. Musawa, Ricky, I. A. Iswanto, and M. F. Hidayat, “Exploring Transformer-Based Model in Sentiment Analysis of Movie Review,” in 2024 8th International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE), Aug. 2024, pp. 214–219. doi: 10.1109/ICITISEE63424.2024.10730492.
[8] X. Zhu, S. Gardiner, T. Roldán, and D. Rossouw, “The Model Arena for Cross-lingual Sentiment Analysis: A Comparative Study in the Era of Large Language Models,” pp. 141–152, 2024, [Online]. Available: https://aclanthology.org/2024.wassa-1.12
[9] P. Xue and W. Bai, “A Fine-Grained Sentiment Analysis Method Using Transformer for Weibo Comment Text,” Int. J. Inf. Technol. Syst. Appoach, vol. 17, no. 1, pp. 1–24, Jul. 2024, doi: 10.4018/IJITSA.345397.
[10] T. Bolukbasi, K.-W. Chang, J. Zou, V. Saligrama, and A. Kalai, “Man is to computer programmer as woman is to homemaker? debiasing word embeddings,” in Proceedings of the 30th International Conference on Neural Information Processing Systems, in NIPS’16. Red Hook, NY, USA: Curran Associates Inc., 2016, pp. 4356–4364.
[11] M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier,” in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, in KDD ’16. New York, NY, USA: Association for Computing Machinery, 2016, pp. 1135–1144. doi: 10.1145/2939672.2939778.
[12] M. Ferrari Dacrema, P. Cremonesi, and D. Jannach, “Are we really making much progress? A worrying analysis of recent neural recommendation approaches,” in Proceedings of the 13th ACM Conference on Recommender Systems, in RecSys ’19. New York, NY, USA: Association for Computing Machinery, 2019, pp. 101–109. doi: 10.1145/3298689.3347058.
[13] R. Stefan, G. Carutasu, and M. Mocan, “Ethical Considerations in the Implementation and Usage of Large Language Models,” in The 17th International Conference Interdisciplinarity in Engineering, L. Moldovan and A. Gligor, Eds., Cham: Springer Nature Switzerland, 2024, pp. 131–144.
[14] J. P. Venugopal, A. A. V. Subramanian, G. Sundaram, M. Rivera, and P. Wheeler, “A Comprehensive Approach to Bias Mitigation for Sentiment Analysis of Social Media Data,” Appl. Sci., vol. 14, no. 23, 2024, doi: 10.3390/app142311471.
[15] K. L. Revathi, A. R. Satish, and P. S. Rao, “Fine - Grained Sentiment Analysis on Online Reviews,” in 2023 Third International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT), Jan. 2023, pp. 1–6. doi: 10.1109/ICAECT57570.2023.10118291.
[16] H. Zheng, J. Zhang, Y. Suzuki, F. Fukumoto, and H. Nishizaki, “Semi-Supervised Learning for Aspect-Based Sentiment Analysis,” in 2021 International Conference on Cyberworlds (CW), 2021, pp. 209–212. doi: 10.1109/CW52790.2021.00042.
[17] A. D. L. Languré and M. Zareei, “Breaking Barriers in Sentiment Analysis and Text Emotion Detection: Toward a Unified Assessment Framework,” IEEE Access, vol. 11, pp. 125698–125715, 2023, doi: 10.1109/ACCESS.2023.3331323.
[18] S. H. Lye and P. L. Teh, “Customer Intent Prediction using Sentiment Analysis Techniques,” in 2021 11th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), 2021, pp. 185–190. doi: 10.1109/IDAACS53288.2021.9660391.
[19] J. Y. M. Nip and B. Berthelier, “Social Media Sentiment Analysis,” Encyclopedia, vol. 4, no. 4, pp. 1590–1598, 2024, doi: 10.3390/encyclopedia4040104.
[20] A. PATEL, P. OZA, and S. AGRAWAL, “Sentiment Analysis of Customer Feedback and Reviews for Airline Services using Language Representation Model,” Procedia Comput. Sci., vol. 218, pp. 2459–2467, 2023, doi: https://doi.org/10.1016/j.procs.2023.01.221.
[21] C. P. Gupta and V. V Ravi Kumar, “Sentiment Analysis and its Application in Analysing Consumer Behaviour,” in 2023 International Conference on Emerging Techniques in Computational Intelligence (ICETCI), 2023, pp. 332–337. doi: 10.1109/ICETCI58599.2023.10331537.
[22] M. Z. Ansari, M. B. Aziz, M. O. Siddiqui, H. Mehra, and K. P. Singh, “Analysis of Political Sentiment Orientations on Twitter,” Procedia Comput. Sci., vol. 167, pp. 1821–1828, 2020, doi: https://doi.org/10.1016/j.procs.2020.03.201.
[23] P. M. Madan, M. R. Madan, and D. P. Thakur, “Analysing The Patient Sentiments in Healthcare Domain Using Machine Learning,” Procedia Comput. Sci., vol. 238, pp. 683–690, 2024, doi: https://doi.org/10.1016/j.procs.2024.06.077.
[24] P. Thangavel and R. Lourdusamy, “A lexicon-based approach for sentiment analysis of multimodal content in tweets,” Multimed. Tools Appl., vol. 82, no. 16, pp. 24203–24226, 2023, doi: 10.1007/s11042-023-14411-3.
[25] A. Briciu, A.-D. Călin, D.-L. Miholca, C. Moroz-Dubenco, V. Petrașcu, and G. Dascălu, “Machine-Learning-Based Approaches for Multi-Level Sentiment Analysis of Romanian Reviews,” Mathematics, vol. 12, no. 3, 2024, doi: 10.3390/math12030456.
[26] R. Jozefowicz, W. Zaremba, and I. Sutskever, “An empirical exploration of recurrent network architectures,” in Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, in ICML’15. JMLR.org, 2015, pp. 2342–2350.
[27] I. Perikos and A. Diamantopoulos, “Explainable Aspect-Based Sentiment Analysis Using Transformer Models,” Big Data Cogn. Comput., vol. 8, no. 11, 2024, doi: 10.3390/bdcc8110141.
[28] M. Kasri, M. Birjali, M. Nabil, A. Beni-Hssane, A. El-Ansari, and M. El Fissaoui, “Refining Word Embeddings with Sentiment Information for Sentiment Analysis,” J. ICT Stand., vol. 10, no. 3, pp. 353–382, 2022, doi: 10.13052/jicts2245-800X.1031.
[29] C. Yue, A. Li, Z. Chen, G. Luan, and S. Guo, “Domain-Aware Neural Network with a Novel Attention-Pooling Technology for Binary Sentiment Classification,” Appl. Sci., vol. 14, no. 17, 2024, doi: 10.3390/app14177971.
[30] W. Li, D. Li, H. Yin, L. Zhang, Z. Zhu, and P. Liu, “Lexicon-Enhanced Attention Network Based on Text Representation for Sentiment Classification,” Appl. Sci., vol. 9, no. 18, 2019, doi: 10.3390/app9183717.
[31] M. Rizinski, H. Peshov, K. Mishev, M. Jovanovik, and D. Trajanov, “Sentiment Analysis in Finance: From Transformers Back to eXplainable Lexicons (XLex),” IEEE Access, vol. 12, pp. 7170–7198, 2024, doi: 10.1109/ACCESS.2024.3349970.
[32] M. Bilal and A. A. Almazroi, “Effectiveness of Fine-tuned BERT Model in Classification of Helpful and Unhelpful Online Customer Reviews,” Electron. Commer. Res., vol. 23, no. 4, pp. 2737–2757, 2023, doi: 10.1007/s10660-022-09560-w.
[33] Z. Li, L. Zhou, X. Yang, H. Jia, W. Li, and J. Zhang, “User Sentiment Analysis of COVID-19 via Adversarial Training Based on the BERT-FGM-BiGRU Model,” Systems, vol. 11, no. 3, 2023, doi: 10.3390/systems11030129.
[34] G. Zhao, Y. Luo, Q. Chen, and X. Qian, “Aspect-based sentiment analysis via multitask learning for online reviews,” Knowledge-Based Syst., vol. 264, p. 110326, 2023, doi: https://doi.org/10.1016/j.knosys.2023.110326.
[35] B. Liang et al., “Few-shot Aspect Category Sentiment Analysis via Meta-learning,” ACM Trans. Inf. Syst., vol. 41, no. 1, Jan. 2023, doi: 10.1145/3529954.
[36] P. Wang, J. Li, and J. Hou, “S2SAN: A sentence-to-sentence attention network for sentiment analysis of online reviews,” Decis. Support Syst., vol. 149, p. 113603, 2021, doi: https://doi.org/10.1016/j.dss.2021.113603.
[37] D. Lee, C. S. Prakash, J. FitzGerald, and J. Lehmann, “MATTER: Memory-Augmented Transformer Using Heterogeneous Knowledge Sources,” 2024, [Online]. Available: http://arxiv.org/abs/2406.04670
[38] B. Liu, W. Guan, C. Yang, Z. Fang, and Z. Lu, “Transformer and Graph Convolutional Network for Text Classification,” Int. J. Comput. Intell. Syst., vol. 16, no. 1, p. 161, 2023, doi: 10.1007/s44196-023-00337-z.
[39] C. Yeh, Y. Chen, A. Wu, C. Chen, F. Viegas, and M. Wattenberg, “AttentionViz: A Global View of Transformer Attention,” IEEE Trans. Vis. Comput. Graph., vol. 30, no. 1, pp. 262–272, 2024, doi: 10.1109/TVCG.2023.3327163.
[40] J.-S. Pan, G.-L. Wang, S.-C. Chu, D. Yang, and V. Snášel, “New feature attribution method for explainable aspect-based sentiment classification,” Knowledge-Based Syst., vol. 304, p. 112550, 2024, doi: https://doi.org/10.1016/j.knosys.2024.112550.
[41] S. Sivakumar and R. Rajalakshmi, “Context-aware sentiment analysis with attention-enhanced features from bidirectional transformers,” Soc. Netw. Anal. Min., vol. 12, no. 1, p. 104, 2022, doi: 10.1007/s13278-022-00910-y.
[42] A. Shirgaonkar, N. Pandey, N. C. Abay, T. Aktas, and V. Aski, “Knowledge Distillation Using Frontier Open-source LLMs: Generalizability and the Role of Synthetic Data,” 2024, [Online]. Available: http://arxiv.org/abs/2410.18588
[43] D. You and D. Chon, “Trust & Safety of LLMs and LLMs in Trust & Safety,” 2024, [Online]. Available: http://arxiv.org/abs/2412.02113
[44] Y. Guo et al., “Bias in Large Language Models: Origin, Evaluation, and Mitigation,” pp. 1–47, 2024, [Online]. Available: http://arxiv.org/abs/2411.10915
[45] X. Fang, S. Che, M. Mao, H. Zhang, M. Zhao, and X. Zhao, “Bias of AI-generated content: an examination of news produced by large language models.,” Sci. Rep., vol. 14, no. 1, p. 5224, Mar. 2024, doi: 10.1038/s41598-024-55686-2.
[46] L. H. X. Ng, I. Cruickshank, and R. K.-W. Lee, “Examining the Influence of Political Bias on Large Language Model Performance in Stance Classification,” 2024, [Online]. Available: http://arxiv.org/abs/2407.17688
[47] G. Kuzmin, N. Yadav, I. Smirnov, T. Baldwin, and A. Shelmanov, “Inference-Time Selective Debiasing to Enhance Fairness in Text Classification Models,” 2024, [Online]. Available: http://arxiv.org/abs/2407.19345
[48] N. Raghunathan and K. Saravanakumar, “Challenges and Issues in Sentiment Analysis: A Comprehensive Survey,” IEEE Access, vol. 11, no. June, pp. 69626–69642, 2023, doi: 10.1109/ACCESS.2023.3293041.
[49] F. Ronchini, L. Comanducci, and F. Antonacci, “Synthetic training set generation using text-to-audio models for environmental sound classification,” 2024, [Online]. Available: http://arxiv.org/abs/2403.17864
[50] D. Sanmartín and V. Prohaska, “Exploring Tpus for Ai Applications,” 2023.
[51] N. Wang, K. Walter, Y. Gao, and A. Abuadbba, “Through the Lens of Attack Objectives,” pp. 1–15.
[52] H. Waghela, “Robust Image Classification : Defensive Strategies against FGSM and PGD Adversarial Attacks”.
Published
2025-01-24
How to Cite
[1]
A. Amrullah, “Sentiment Analysis in the Age of Transformers and Large Language Models: A Comprehensive Review of Recent Advances and Future”, Intellithings Journal, vol. 1, no. 1, pp. 39-49, Jan. 2025.
Section
Articles
