基于双注意力卷积神经网络模型的情感分析研究

doi:10.12052/gdutxb.190042

摘要/Abstract

摘要： 卷积神经网络（Convolutional Neural Networks，CNN）无法判别输入文本中特征词与情感的相关性.因此提出一种双注意力机制的卷积神经网络模型（Double Attention Convolutional Neural Networks，DACNN），将词特征与词性特征有效融合后得到本文的特征表示，确定情感倾向.本文提出局部注意力的卷积神经网络模型，改进卷积神经网络的特征提取能力，采用双通道的局部注意力卷积神经网络提取文本的词特征和词性特征.然后使用全局注意力为特征分配不同的权重，有选择地进行特征融合，最后得到文本的特征表示.将该模型在MR和SST-1数据集上进行验证，较普通卷积神经网络和传统机器学习方法，在准确率上分别取得0.7%和1%的提升.

关键词: 卷积神经网络, 注意力机制, 情感分类

Abstract: Convolutional Neural Networks (CNN) cannot discriminate the correlation between feature words and emotions in the input text. Therefore, this paper proposes a double attention mechanism convolutional neural network to incorporate word and POS(Part-of-speech Tagging) information into sentiment classification (Double Attention Convolutional Neural Networks, DACNN), which combines the word features with the part-of-speech features to obtain the feature representation of the text and determine the sentiment tendency. In this paper, we introduce an attention convolutional neural network to incorporate word and POS information into sentiment classification. The first local attention mechanism captures local feature from word and POS channels. And then we propose a novel architect for combining two features by a global attention mechanism, which can decide how much information to use from word or POS information. The model is validated on MR and SST-1 datasets, achieving 0.7% and 1% improvement in accuracy compared to conventional convolutional neural networks and traditional machine learning methods.

Key words: convolutional neural networks, attention mechanism, sentiment classification

中图分类号:

TP391

曾碧卿, 韩旭丽, 王盛玉, 徐如阳, 周武. 基于双注意力卷积神经网络模型的情感分析研究[J]. 广东工业大学学报, 2019, 36(04): 10-17.

Zeng Bi-qing, Han Xu-li, Wang Sheng-yu, Xu Ru-yang, Zhou Wu. Sentiment Classification Based on Double Attention Convolutional Neural Network Model[J]. Journal of Guangdong University of Technology, 2019, 36(04): 10-17.

参考文献

[1] PANG B, LEE L, VAITHYANATHAN S. Thumbs up:sentiment classification using machine learning techniques[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10. Stroudsburg:ACM. 2002:79-86.
[2] PEREZ-ROSAZ V, BANEA C, MIHALCEA R. Learning sentiment lexicons in spanish[C]//LREC. Istanbul:ELRA. 2012, 12:73.
[3] KIRITCHENKO S, ZHU X, MOHAMMAD S M. Sentiment analysis of short informal text[J]. Journal of Artificial Intelligence Research, 2014, 50:723-762
[4] WANG S, MANNING C D. Baselines and bigrams:simple, good sentiment and topic classification[C]//Meeting of the Association for Computational Linguistics:Short Papers. Jeju Island:ACL, 2012:90-94.
[5] ZHANG L, WANG S, LIU B. Deep learning for sentiment analysis:a survey[J]. Wiley Interdisciplinary Reviews:Data Mining and Knowledge Discovery, 2018, 8(4):e1253
[6] 张玥杰, 徐智婷, 薛向阳. 融合多特征的最大熵汉语命名实体识别模型[J]. 计算机研究与发展, 2008, 45(6):1004-1010 ZHANG Y J, XU Z T, XUE X Y. Fusion of multiple features for chinese named entity recognition based on maximum entropy model[J]. Journal of Computer Research and Development, 2008, 45(6):1004-1010
[7] CHEN P, SUN Z, BING L, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen:ACL, 2017:452-461.
[8] WU Y, SCHUSTER M, CHEN Z, et al. Google's neural machine translation system:Bridging the gap between human and machine translation[J]. arXiv preprint arXiv:1609.08144, 2016.
[9] YIN W, SCHÜTZE H, XIANG B, et al. Abcnn:attention-based convolutional neural network for modeling sentence pairs[J]. Transactions of the Association for Computational Linguistics, 2016, 4:259-272
[10] GOLUB D, HE X. Character-level question answering with attention[J]. arXiv preprint arXiv:1604.00727, 2016
[11] RUSH A M, CHOPRA S, WESTON J. A neural attention model for abstractive sentence summarization[J]. arXiv preprint arXiv:1509.00685, 2015.
[12] REI M, CRICHTON G K O, PYYSALO S. Attending to characters in neural sequence labeling models[J]. arXiv preprint arXiv:1611.04361, 2016.
[13] KIM Y. Convolutional neural networks for sentence classification[J]. arXiv preprint arXiv:1408.5882, 2014
[14] KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences[J]. arXiv preprint arXiv:1404.2188, 2014.
[15] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is All you Need[J]. Neural information processing systems, 2017:5998-6008
[16] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Advances in Neural Information Processing Systems. Lake tahoe Nevada:NIPS, 2013:3111-3119.
[17] BENGIO Y, DUCHARME R, VINCENT P, et al. A neural probabilistic language model[J]. Journal of machine learning research, 2003, 3(6):1137-1155
[18] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[J]. arXiv preprint arXiv:1301.3781, 2013.
[19] 梁斌, 刘全, 徐进, 等. 基于多注意力卷积神经网络的特定目标情感分析[J]. 计算机研究与发展, 2017, 54(8):1724-1735 LIANG B, LIU Q, XU J, et al. Aspect-based sentiment analysis based on multi-attention CNN[J]. Journal of Computer Research and Development, 2017, 54(8):1724-1735
[20] ZHANG Y, WALLACE B. A sensitivity analysis of (and practitioners' guide to) convolutional neural networks for sentence classification[J]. arXiv preprint arXiv:1510.03820, 2015.
[21] SOCHER R, PERELYGIN A, WU J, et al. Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Seattle, Washington:ACL, 2013:1631-1642.
[22] TAI K S, SOCHER R, MANNING C D. Improved semantic representations from tree-structured long short-term memory networks[J]. arXiv preprint arXiv:1503.00075, 2015.
[23] TANG D, QIN B, LIU T. Document modeling with gated recurrent neural network for sentiment classification[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Lisbon, Portugal:ACL, 2015:1422-1432.
[24] BHATIA P, JI Y, EISENSTEIN J. Better document-level sentiment analysis from rst discourse parsing[J]. arXiv preprint arXiv:1509.01599, 2015.
[25] XU K, BA J, KIROS R, et al. Show, attend and tell:Neural image caption generation with visual attention[C]//International Conference on Machine Learning. Lille, France:ACM, 2015:2048-2057.
[26] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[J]. arXiv preprint arXiv:1409.0473, 2014.
[27] LUONG M T, PHAM H, MANNING C D. Effective approaches to attention-based neural machine translation[J]. arXiv preprint arXiv:1508.04025, 2015.
[28] YANG Z, YANG D, DYER C, et al. Hierarchical attention networks for document classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Diego, California:ACL. 2016:1480-1489.
[29] WANG L, CAO Z, MELO G D, et al. Relation classification via multi-level attention CNNs[C]//Meeting of the Association for Computational Linguistics. Berlin:ACL, 2016:1298-1307.
[30] WANG S, MANNING C D. Baselines and bigrams:simple, good sentiment and topic classification[C]//Meeting of the Association for Computational Linguistics:Short Papers. Association for Computational Linguistics, Jeju Island:ACL, 2012:90-94.
[31] SHIN B, LEE T, CHOI J D, et al. Lexicon integrated CNN models with attention for sentiment analysis[J]. Empirical Methods in Natural Language Processing, 2017:149-158
[32] LE Q V, MIKOLOV T. Distributed representations of sentences and documents[J]. International Conference on Machine Learning, 2014:1188-1196
[33] DONG L, WEI F, LIU S, et al. A statistical parsing framework for sentiment classification[J]. Computational Linguistics, 2014, 41(2):293-336

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed