As a significant branch of natural language processing, deep learning-based sentiment analysis has dominated in text sentiment analysis instead of lexicon graph and machine learning methods. To further improve the quality of sentiment analysis, we propose a hybrid sentiment analysis method of transformer and capsule network for hotel reviews. The proposed approach takes advantages of both self-attention mechanism in transformer and detailed representation in capsule network to capture bidirectional semantic features well. Compared with the traditional RNN, CNN and pure transformer, the hybrid sentiment analysis method of transformer and capsule network performs 13.83%, 8.97%, and 8.02% higher accuracy for an open-source dataset of hotel reviews respectively. The comprehensive experiments results demonstrate that our proposed method achieves higher quality of sentiment analysis than latest methods. 

Yanxiang Li, Jiajia Jiao. A Hybrid Sentiment Analysis Method of Transformer and Capsule Network for Hotel Reviews. Academic Journal of Computing & Information Science (2021), Vol. 4, Issue 1: 60-65. https://doi.org/10.25236/AJCIS.2021.040110.

[1] Zhang L, Wang S, Liu B. Deep Learning for Sentiment Analysis: A Survey [J]. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2018, 8(4): 1253-1287.

[2] MEI L L, HUANG H Y, ZHOU X Y, MAO X L. An overview of sentiment lexicon construction [J]. Journal of Chinese Information Processing, 2016, 30(5): 19-27.

[3] Schuller B, Mousa E D, Vryniotis V. Sentiment analysis and opinion mining [J]. Wiley Interdisciplinary Reviews Data Mining & Knowledge Discovery, 2015, 5(5): 255-263.

[4] HONG W, LI M. A review of textual sentiment analysis methods [J].Computer Engineering and Science, 2019, 41(04): 750-757.

[5] DU H, XU X K, WU D Y, LIU Y, YU Z H, CHENG X Q. A Sentiment Classification Method Based on Sentiment-Specific Word Embedding [J]. Journal of Chinese Information Processing, 2017, 31(3): 170-176.

[6] MIAO G H. Emotion Mining and Simulation Analysis of Microblogging Based on Word2vec and SVM [J]. Electronic Science and Technology, 2018, 31(5): 81-83.

[7] CEHNG Z S, WANG L. Sentiment analysis method of network comments based on support vector machine [J]. Electronic Technology & Software Engineering. 2019, 162(16): 19-20.

[8] SHEN Y L, ZHAO X B. A Review of Research on Aspect-based Sentiment Analysis based on Deep Learning [J]. Information Technology & Standardization, 2020, (Z1): 50-53+58.

[9] PING Y N. Fine-grained Sentiment Analysis of Multi-source News Comments Based on Capsule Network [D].Shanghai: Shanghai Normal University, 2020.

[10] Kim Y. Convolutional Neural Networks for Sentence Classification [J]. Eprint Arxiv, 2014.

[11] Bahdanau D, Cho K, Bengio Y. Neural Machine Translation by Jointly Learning to Align and Translate [J]. Computer Science, 2014.

[12] Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need [J]. arXiv, 2017.

[13] WU Y W, HANG K, WANG X Y, et al. Method of Emotional Classification in Short Texts Combined with LDA Models [J]. Journal of Chinese Computer Systems, 2019, 40(10). 2082-2086.

[14] WANG J Q, GONG Z H, XUE Y, PANG S G, GU D H. Target-specific Sentiment Analysis Based on Mixed Multi-head Attention and Capsule Network [J]. Journal of Chinese Information Processing, 2020, 34(5): 100-110.

[15] SABOURS, FROSSTN, HINTONGE. Dynamic routing between capsules [C]. Advances in Neural Information Processing Systems. 2017: 3856-3866.

[16] LIN Y, QIAN T Y. Cross-domain Sentiment Classification Based on Capsule Network [J]. Journal of Nanjing University of Information Science & Technology (Natural Science Edition), 2019, 11(03): 286-294.

[17] XU L. Short Text Sentiment Analysis Based on Self-attention and Capsule Network [J]. Computer and Modernization, 2020 (07): 61-64+70.

[18] ZHONG M H, HUANG H. An analysis of the connotation of socialist core values [J]. Shandong Social Sciences, 2009 (12): 14-18.

[19] HU Y J, JIANG J X, CHANG H Y. A New Method of Keywords Extraction for Chinese Short – text Classification [J]. Data Analysis and Knowledge Discovery, 2013 (6): 42-48.

[20] HINTONGE, SABOURS, FROSSTN. Matrix capsules with EM routing [C]. The 6th International Conference on Learning Representations (ICLR). 2018: 789.

[21] LIU L F, YANG L, ZHANG S W, LIN H F. Microblog Sentiment Analysis Based on Convolutional Neural Network [J]. Journal of Chinese Information Processing, 2015, 29(6): 159-165.