基于双注意力卷积神经网络模型的情感分析研究

doi:10.12052/gdutxb.190042

Abstract

Abstract: Convolutional Neural Networks (CNN) cannot discriminate the correlation between feature words and emotions in the input text. Therefore, this paper proposes a double attention mechanism convolutional neural network to incorporate word and POS(Part-of-speech Tagging) information into sentiment classification (Double Attention Convolutional Neural Networks, DACNN), which combines the word features with the part-of-speech features to obtain the feature representation of the text and determine the sentiment tendency. In this paper, we introduce an attention convolutional neural network to incorporate word and POS information into sentiment classification. The first local attention mechanism captures local feature from word and POS channels. And then we propose a novel architect for combining two features by a global attention mechanism, which can decide how much information to use from word or POS information. The model is validated on MR and SST-1 datasets, achieving 0.7% and 1% improvement in accuracy compared to conventional convolutional neural networks and traditional machine learning methods.

Key words: convolutional neural networks, attention mechanism, sentiment classification

CLC Number:

TP391

Zeng Bi-qing, Han Xu-li, Wang Sheng-yu, Xu Ru-yang, Zhou Wu. Sentiment Classification Based on Double Attention Convolutional Neural Network Model[J].Journal of Guangdong University of Technology, 2019, 36(04): 10-17.

References

[1] PANG B, LEE L, VAITHYANATHAN S. Thumbs up:sentiment classification using machine learning techniques[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10. Stroudsburg:ACM. 2002:79-86.
[2] PEREZ-ROSAZ V, BANEA C, MIHALCEA R. Learning sentiment lexicons in spanish[C]//LREC. Istanbul:ELRA. 2012, 12:73.
[3] KIRITCHENKO S, ZHU X, MOHAMMAD S M. Sentiment analysis of short informal text[J]. Journal of Artificial Intelligence Research, 2014, 50:723-762
[4] WANG S, MANNING C D. Baselines and bigrams:simple, good sentiment and topic classification[C]//Meeting of the Association for Computational Linguistics:Short Papers. Jeju Island:ACL, 2012:90-94.
[5] ZHANG L, WANG S, LIU B. Deep learning for sentiment analysis:a survey[J]. Wiley Interdisciplinary Reviews:Data Mining and Knowledge Discovery, 2018, 8(4):e1253
[6] 张玥杰, 徐智婷, 薛向阳. 融合多特征的最大熵汉语命名实体识别模型[J]. 计算机研究与发展, 2008, 45(6):1004-1010 ZHANG Y J, XU Z T, XUE X Y. Fusion of multiple features for chinese named entity recognition based on maximum entropy model[J]. Journal of Computer Research and Development, 2008, 45(6):1004-1010
[7] CHEN P, SUN Z, BING L, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen:ACL, 2017:452-461.
[8] WU Y, SCHUSTER M, CHEN Z, et al. Google's neural machine translation system:Bridging the gap between human and machine translation[J]. arXiv preprint arXiv:1609.08144, 2016.
[9] YIN W, SCHÜTZE H, XIANG B, et al. Abcnn:attention-based convolutional neural network for modeling sentence pairs[J]. Transactions of the Association for Computational Linguistics, 2016, 4:259-272
[10] GOLUB D, HE X. Character-level question answering with attention[J]. arXiv preprint arXiv:1604.00727, 2016
[11] RUSH A M, CHOPRA S, WESTON J. A neural attention model for abstractive sentence summarization[J]. arXiv preprint arXiv:1509.00685, 2015.
[12] REI M, CRICHTON G K O, PYYSALO S. Attending to characters in neural sequence labeling models[J]. arXiv preprint arXiv:1611.04361, 2016.
[13] KIM Y. Convolutional neural networks for sentence classification[J]. arXiv preprint arXiv:1408.5882, 2014
[14] KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences[J]. arXiv preprint arXiv:1404.2188, 2014.
[15] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is All you Need[J]. Neural information processing systems, 2017:5998-6008
[16] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Advances in Neural Information Processing Systems. Lake tahoe Nevada:NIPS, 2013:3111-3119.
[17] BENGIO Y, DUCHARME R, VINCENT P, et al. A neural probabilistic language model[J]. Journal of machine learning research, 2003, 3(6):1137-1155
[18] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv:1301.3781, 2013.
[19] 梁斌, 刘全, 徐进, 等. 基于多注意力卷积神经网络的特定目标情感分析[J]. 计算机研究与发展, 2017, 54(8):1724-1735 LIANG B, LIU Q, XU J, et al. Aspect-based sentiment analysis based on multi-attention CNN[J]. Journal of Computer Research and Development, 2017, 54(8):1724-1735
[20] ZHANG Y, WALLACE B. A sensitivity analysis of (and practitioners' guide to) convolutional neural networks for sentence classification[J]. arXiv preprint arXiv:1510.03820, 2015.
[21] SOCHER R, PERELYGIN A, WU J, et al. Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle, Washington:ACL, 2013:1631-1642.
[22] TAI K S, SOCHER R, MANNING C D. Improved semantic representations from tree-structured long short-term memory networks[J]. arXiv preprint arXiv:1503.00075, 2015.
[23] TANG D, QIN B, LIU T. Document modeling with gated recurrent neural network for sentiment classification[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal:ACL, 2015:1422-1432.
[24] BHATIA P, JI Y, EISENSTEIN J. Better document-level sentiment analysis from rst discourse parsing[J]. arXiv preprint arXiv:1509.01599, 2015.
[25] XU K, BA J, KIROS R, et al. Show, attend and tell:Neural image caption generation with visual attention[C]//International Conference on Machine Learning. Lille, France:ACM, 2015:2048-2057.
[26] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv:1409.0473, 2014.
[27] LUONG M T, PHAM H, MANNING C D. Effective approaches to attention-based neural machine translation[J]. arXiv preprint arXiv:1508.04025, 2015.
[28] YANG Z, YANG D, DYER C, et al. Hierarchical attention networks for document classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Diego, California:ACL. 2016:1480-1489.
[29] WANG L, CAO Z, MELO G D, et al. Relation classification via multi-level attention CNNs[C]//Meeting of the Association for Computational Linguistics. Berlin:ACL, 2016:1298-1307.
[30] WANG S, MANNING C D. Baselines and bigrams:simple, good sentiment and topic classification[C]//Meeting of the Association for Computational Linguistics:Short Papers. Association for Computational Linguistics, Jeju Island:ACL, 2012:90-94.
[31] SHIN B, LEE T, CHOI J D, et al. Lexicon integrated CNN models with attention for sentiment analysis[J]. Empirical Methods in Natural Language Processing, 2017:149-158
[32] LE Q V, MIKOLOV T. Distributed representations of sentences and documents[J]. International Conference on Machine Learning, 2014:1188-1196
[33] DONG L, WEI F, LIU S, et al. A statistical parsing framework for sentiment classification[J]. Computational Linguistics, 2014, 41(2):293-336

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Sentiment Classification Based on Double Attention Convolutional Neural Network Model

HTML

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 7

Metrics

Comments

Recommended 0

[1]	Wu Jun-xian, He Yuan-lie. Channel Attentive Self-supervised Network for Monocular Depth Estimation [J]. Journal of Guangdong University of Technology, 2023, 40(02): 22-29.
[2]	Liu Hong-wei, Lin Wei-zhen, Wen Zhan-ming, Chen Yan-jun, Yi Min-qi. A MABM-based Model for Identifying Consumers' Sentiment Polarity―Taking Movie Reviews as an Example [J]. Journal of Guangdong University of Technology, 2022, 39(06): 1-9.
[3]	Teng Shao-hua, Dong Pu, Zhang Wei. An Attention Text Summarization Model Based on Syntactic Structure Fusion [J]. Journal of Guangdong University of Technology, 2021, 38(03): 1-8.
[4]	Liang Guan-shu, Cao Jiang-zhong, Dai Qing-yun, Huang Yun-fei. An Unsupervised Trademark Retrieval Method Based on Attention Mechanism [J]. Journal of Guangdong University of Technology, 2020, 37(06): 41-49.
[5]	Zhan Yin-wei, Zhu Bai-wan, Yang Zhuo. Research and Application of Vehicle Color and Model Recognition Algorithm [J]. Journal of Guangdong University of Technology, 2020, 37(04): 9-14.
[6]	Gao Jun-yan, Liu Wen-yin, Yang Zhen-guo. Object Tracking Combined with Attention and Feature Fusion [J]. Journal of Guangdong University of Technology, 2019, 36(04): 18-23.
[7]	CHEN Pei-Wen, FU Xiu-Fen. Research on Sentiment Classification of Texts Based on SVM [J]. Journal of Guangdong University of Technology, 2014, 31(3): 95-101.