Journal of Guangdong University of Technology ›› 2024, Vol. 41 ›› Issue (01): 69-78.doi: 10.12052/gdutxb.220132

• Computer Science and Technology • Previous Articles     Next Articles

Short Text Feature Extension and Classification Method Based on Semantic Embedding of Tags and Graph Convolution Network

Zhang Ling1, Li Rong-zhen1, Zheng Su2   

  1. 1. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China;
    2. College of Education, Ningxia University, Yinchuan 750001, China
  • Received:2022-08-29 Online:2024-01-25 Published:2023-11-20

Abstract: In short text classification, too short text length, fewer keywords and underutilization of the label information leads to the severe problems of sparse features and ambiguous semantics, which can affect the performance of short text classification. Agraph convolution network model based on tag semantic embedding is proposed for the problem. Firstly, according to TF/IDF, a new word frequency method is proposed, which comprehensively considers the inter-class and intra-class distribution of words in the global corpus. Then, through By label embedding method, each training text with the corresponding label is mapped into one feature space in the text graph. After filtering and aggregation in one feature space, synonyms embedded of label information can highlight the category representation. Finally, the text graph is input into the graph convolution neural network to learn new feature. Both the learned new feature Both the learned new feature and the features from the pre-training model can improve the classification accuracy of short texts and the generalization ability of the whole model. We choose four short text datasets such as MR, web_snippets, R8 and R52, to evaluate the performance of our proposed algorithm and fourteen benchmark models. The experimental results show that the proposed model in this paper is superior to others in classification accuracy, recall ratio and F1-score.

Key words: short text, semantics of label, feature space, graph convolution network, pre-training model

CLC Number: 

  • TP391
[1] ZAREMBA W, SUTSKEVER I, VINYALS O. Recurrent neural network regularization[EB/OL]. (2014-09-08) [2022-08-20]. https://arxiv.org/abs/1409.2329v5.
[2] TURKOGLU M, HANBAY D, SENGUR A. Multi-model L- STM-based convolutional neural networks for detection of apple diseases and pests [J]. Journal of Ambient Intelligence and Humanized Computing, 2022, 13(1): 3335-3345.
[3] AGARAP A F. A neural network architecture combining gated recurrent unit (GRU) and support vector machine(SVM) for intrusion detection in network traffic data[EB/OL]. (2017-09-10) [2022-08-20]. https://arxiv.org/abs/1709.0302.
[4] CANIZO M, TRIGUERO I, CONDE A, et al. Multi-head CNN-RNN for multi-time series anomaly detection: an ind- ustrial case study [J]. Neurocomputing, 2019, 363: 246-260.
[5] KIM Y. Convolutional neural network for sentence classification[EB/OL]. arXiv: 1408.5882 (2014-09-03) [2022-08-20]. https://arxiv.org/abs/1408.5882.
[6] NIEPERT M, AHMED M, KUTZKOV K. Learning convolutional neural networks for graphs[EB/OL]. arXiv: 1605.05273(2016-06-08) [2022-08-20]. https://arxiv.org/abs/1605.05273.
[7] XU K, HU W, LESKOVEC J, et al. How powerful are graph neural networks?[C]//International Conference on Learning Representation. New Orleans: ICLR, 2019: 1-17.
[8] KIPF, WELLING M, THOMAS N. Semi-supervisedclassification with graph convolutional networks[C]//International Conference on Learning Representations. Toulon, France: ICLR, 2017: 1-14.
[9] YAO L, MAO C S, LUO Y. Graph convolutional networks for text classification[C]//33rd AAAI Conference on Artificial Intelligence. Honolulu: AAAI, 2019: 7370-7377.
[10] DRASKO R, BOZO K. Review spam detection using mac- hine learning[C]//23th International Scientific-Professional Conference on Information Technology. New Delhi, India: IT, 2018: 1-4.
[11] BAKSHI R K, KAUR N, KAUR R, et al. Opinion mining and sentiment analysis[C]//Computing for Sustainable Glo-bal Development. New Delhi, India: INDIACom, 2016: 452-455.
[12] BOUAZIZ A, DARTIGUES-PALLEZ C, PEREIRA C D C, et al. Short text classification using semantic random forest [J]. Springer International Publishing, 2014, 8646: 288-299.
[13] 方澄, 李贝, 韩萍. 基于全局特征图的半监督微博文本情感分类[J]. 信号处理, 2021, 37(6): 1066-1074.
FANG C, LI B, HAN P. Semi-supervised microblog text sentiment classification based on global feature graph [J]. Journal of Signal Processing, 2021, 37(6): 1066-1074.
[14] 崔婉秋, 杜军平, 寇菲菲, 等. 面向微博短文本的社交与概念化语义扩展搜索方法[J]. 计算机研究与发展, 2018, 55(8): 1641-1652.
CUI W Q, DU J P, KOU F F, et al. The social and conceptual semantic extended search method for microblog short text [J]. Journal of Computer Reasearch and Development, 2018, 55(8): 1641-1652.
[15] WANG G, LI C, WANG W, et al. Joint embedding of words and labels for text classification[C]//Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans: NAACL-HLT, 2018: 461-469.
[16] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Conference and Workshop on Neural Information Processing Systems. Long Beach, California, U- SA: ACM, 2017: 6000-6010.
[17] 张万杰. 引入标签语义信息的多标签文本分类[J]. 计算机应用, 2021, 8: 1672-9528.
[18] JI F, YANG J L, ZHANG Q, et al. GraphFlow: a new graph convolutional network based on parallel flows [C]//Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: ICASSP, 2020: 3332-3336.
[19] 郑诚, 董春阳, 黄夏炎. 基于BTM图卷积网络的短文本分类方法[J]. 计算机工程与应用, 2021, 57(4): 155-160.
ZHENG C, DONG C Y, HUANG X Y. Short text classification method based on BTM graph convolution network [J]. Computer Engineering and Application, 2021, 57(4): 155-160.
[20] 辛媛. 基于图神经网络的单标签文本分类[D]. 合肥: 中国科技技术大学, 2021, 1-61.
[21] 申艳光, 贾耀清. 基于词共现与图卷积的文本分类方法[J]. 计算机工程与应用, 2021, 57(11): 173-178.
SHEN Y G, JIA Y Q. Text categorization method based on word co-occurrence and graph convolution [J]. Computer Engineering and Application, 2021, 57(11): 173-178.
[22] 郑诚, 陈杰, 董春阳. 结合图卷积的深层神经网络用于文本分类[J]. 计算机工程与应用, 2022, 58(7): 206-212.
ZHENG C, CHEN J, DONG C Y. Deep neural network combined with graph convolution for text classification [J]. Computer Engineering and Application, 2022, 58(7): 206-212.
[23] LIU X, YOU X, ZHANG X, et al. Tensor graph convolutional networks for text classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York, US-A: AAAI, 2020: 8409-8416.
[24] WU F, ZHANG T, SOUZA A, et al. Simplifying graph convolutional networks[C]//International Conference on Machine Learning. Long Beach, CA, USA: ICML, 2019: 1-14.
[25] ZHANG Y F, YU X L, CUI ZY, et al. Every document owns its structure: inductive text classification via graph neural networks[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Seattle, Washington, United States: ACL, 2020: 334-339.
[26] LI Y J, TARLOW D, BROCKSCHMIDT M, et al. Gated graph sequence neural networks[EB/OL]. arXiv: 1511.05493(2017-09-22) [2022-08-20]. https://arxiv.org/abs/1511.05493.
[27] ROMERO A, CUCURULL G, CASANOVA A, et al. Graph attention networks[EB/OL]. arXiv:1710.10903(2018-02-04) [2022-08-20]. https://arxiv.org/abs/1710.10903.
[28] DING K Z, WANG J L, LI J D, et al. Be more with less: hypergraph attention networks for inductive text classification [C]//The 2020 Conference on Empirical Methods in Natural Language Processing. Online: EMNLP, 2020: 4927-4936.
[29] LIN Y X, MENG Y X, SUN X F, et al. BertGCN: transductive text classification by combining GCN an BERT[C]//Annual Meeting of the Association for Computational Linguistics. Bangkok, Thailand: ACL-IJCNLP, 2021: 1456-1462.
[1] Liang Yu-chen, Cai Nian, Ouyang Wen-sheng, Xie Yi-ying, Wang Ping. CT Diagnosis of Chronic Obstructive Pulmonary Disease Based on Slice Correlation Information [J]. Journal of Guangdong University of Technology, 2024, 41(01): 27-33.
[2] Chen Rui, Cai Nian, Luo Zhi-hao, Liu Xuan, Li Jian. Individual Survival Analysis of Breast Cancer Based on Multi-task Recurrent Neural Network Banded Regression Model [J]. Journal of Guangdong University of Technology, 2024, 41(01): 34-40.
[3] Yang Zhen-xiong, Tan Tai-zhe. Low Illumination Image Enhancement Algorithm Based on Generative Adversarial Network [J]. Journal of Guangdong University of Technology, 2024, 41(01): 55-62.
[4] Kuang Yong-nian, Wang Feng. Video Frame Anomaly Behavior Detection Based on Foreground Area Generative Adversarial Networks [J]. Journal of Guangdong University of Technology, 2024, 41(01): 63-68,92.
[5] Liu Jin-neng, Xiao Yan-shan, Liu Bo. A Least Squares Twin Support Vector Machine Method with Uncertain Data [J]. Journal of Guangdong University of Technology, 2024, 41(01): 79-85.
[6] Zeng An, Chen Xu-zhou, Ji Yu-Zhu, Pan Dan, Xu Xiao-Wei. Cardiac Multiclass Segmentation Method Based on Self-attention and 3D Convolution [J]. Journal of Guangdong University of Technology, 2023, 40(06): 168-175.
[7] Wu Xiao-ling, Chen Xiang-wang, Zhan Wen-tao, Ling Jie. Chinese Medical Named Entity Recognition Based on Gated Attention Unit [J]. Journal of Guangdong University of Technology, 2023, 40(06): 176-184.
[8] Gan Meng-kun, Zeng An, Zhang Xiao-bo. Aortic Re-coarctation Prediction Research Based on Swin-Unet [J]. Journal of Guangdong University of Technology, 2023, 40(05): 34-40.
[9] Zheng Yu, Cai Nian, Ouyang Wen-sheng, Xie Yi-ying, Wang Ping. Super-resolution Segmentation of Hepatobiliary Ducts Based on Deep Correlation Mechanism [J]. Journal of Guangdong University of Technology, 2023, 40(05): 41-46.
[10] Wu Zhen-hua, Tang Wen-yan, Lyu Wen-ge, Chen Ru-jie, Hou Meng-hua, Li De-yuan. Fast Image Segmentation with Multilevel Threshold of Two-dimensional Entropy Based on ISSA and Integral Graph [J]. Journal of Guangdong University of Technology, 2023, 40(05): 47-55.
[11] Li Yang, Zhou Ying. Differential Privacy Trajectory Data Publishing Based on Orientation Control [J]. Journal of Guangdong University of Technology, 2023, 40(05): 56-63.
[12] Dai Bin, Zeng Bi, Wei Peng-fei, Huang Yong-jian. A Task-oriented Dialogue Policy Learning Method of Improved Discriminative Deep Dyna-Q [J]. Journal of Guangdong University of Technology, 2023, 40(04): 9-17,23.
[13] Zhong Geng-jun, Li Dong. A Channel-splited Based Dual-branch Block for 3D Point Cloud Processing [J]. Journal of Guangdong University of Technology, 2023, 40(04): 18-23.
[14] Zhang Jia-yue, Zhang Ling. Knowledge Distillation Method Based on Incremental Class Activation Knowledge [J]. Journal of Guangdong University of Technology, 2023, 40(04): 24-30,36.
[15] Wu Ya-di, Chen Ping-hua. A Music Recommendation Model Based on Users' Long and Short Term Preferences and Music Emotional Attention [J]. Journal of Guangdong University of Technology, 2023, 40(04): 37-44.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!