Journal of Guangdong University of Technology ›› 2023, Vol. 40 ›› Issue (06): 176-184.doi: 10.12052/gdutxb.230065

• Artifical Intelligence • Previous Articles    

Chinese Medical Named Entity Recognition Based on Gated Attention Unit

Wu Xiao-ling, Chen Xiang-wang, Zhan Wen-tao, Ling Jie   

  1. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China
  • Received:2023-05-06 Online:2023-11-25 Published:2023-11-08

HTML

PDF (PC)

658

Abstract: The medical named entity recognition aims to automatically identify and classify medical entities in electronic medical records, which plays a very important role in downstream tasks such as information retrieval and knowledge graph. Existing methods usually ignore the dependencies between entities. To address this, this paper proposes a gated attention unit-based model for Chinese medical named entity recognition. First, the proposed model uses the pre-training model MC-BERT to capture contextual information. Then, it uses the cross-attention and gated attention unit to enhance the interaction between entity query and contextual semantics, and further extract the dependency and correlation between entities. Finally, the proposed model uses the matching algorithm of bipartite graph to calculate the loss. This paper conducted experiments on three datasets, including the CMeEE, CMQNN, and MSRA. The experimental results show that the F1 values of the proposed model on the three datasets are 70.74%, 96.92%, and 95.53%, respectively, which outperforms other related models, demonstrating the effectiveness of the proposed model in Chinese medical named entity recognition task.

Key words: Chinese named entity recognition, gated attention unit, cross attention, MC-BERT

CLC Number: 

  • TP391.41
[1] 苏攀予, 黄健文. 基于知识图谱的BIM技术在工程造价领域的应用研究[J]. 广东工业大学学报, 2023, 40(2): 103-110.
SU P Y, HUANG J W. The application of BIM technology in the field of engineering cost based on knowledge graph [J]. Journal of Guangdong University of Technology, 2023, 40(2): 103-110.
[2] 曹智雄, 吴晓鸰, 骆晓伟, 等. 融合迁移学习与YOLOv5的安全帽佩戴检测算法[J]. 广东工业大学学报, 2023, 40(4): 67-76.
CAO Z X, WU X L, LUO X W, et al. Helmet wearing detection algorithm integrating transfer learning and YOLOv5 [J]. Journal of Guangdong University of Technology, 2023, 40(4): 67-76.
[3] WU S H, SHEN Y L, TAN Z Q, et al. Propose-and-refine: a two-stage set prediction network for nested named entity recognition[C]//Proceedings of the 31th International Joint Conference on Artificial Intelligence. Montreal: IJCAI, 2022: 4418-4424.
[4] WANG B L, LU W. Combining spans into entities: a neural two-stage approach for recognizing discontiguous entities[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the Ninth International Joint Conference on Natural Language Processing. Hong Kong: ACL, 2019: 6217-6225.
[5] LI F, LIN Z C, ZHANG M S, et al. A span-based model for joint overlapped and discontinuous named entity recognition[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Bangkok: ACL, 2021: 4814-4828.
[6] GU Y J, QU X Y, WANG Z F, et al. Delving deep into regularity: a simple but effective method for Chinese named entity recognition[C]//Findings of the Association for Computational Linguistics: NAACL 2022. Seattle: ACL, 2022: 1863-1873.
[7] LUAN Y, WADDEN D, HE L H, et al. A general framework for information extraction using dynamic span graphs[C]//Proceedings of the 2019 Conference of the North American Chapter of the ACL. Florence: ACL, 2019: 3036-3046.
[8] YU J T, BOHNET B, POESIO M. Named entity recognition as dependency parsing[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Seattle: ACL, 2020: 6470-6476.
[9] WANG Q, ZHOU Y M, RUAN T, et al. Incorporating dictionaries into deep neural networks for the Chinese clinical named entity recognition [J]. Journal of Biomedical Informatics, 2019, 92: 103133.
[10] LI X Y, ZHANG H, ZHOU X H. Chinese clinical named entity recognition with variant neural structures based on BERT methods [J]. Journal of Biomedical Informatics, 2020, 107: 103422.
[11] LI X Y, FENG J R, MENG Y X, et al. A unified MRC framework for named entity recognition[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Seattle: ACL, 2020: 5849-5859.
[12] PAN Y, XIN C, SUN Z Y, et al. Enhanced language representation with label knowledge for span extraction[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana: ACL, 2021: 4623-4635.
[13] HANG Y, GUI T, DAI J Q, et al. A unified generative framework for various NER subtasks[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Bangkok: ACL, 2021: 5808-5822.
[14] SHEN Y L, WANG X B, TAN Z Q, et al. Parallel instance query network for named entity recognition[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Dublin: ACL, 2022: 947-961.
[15] LI X N, YAN H, QIU X P, et al. FLAT: Chinese NER using flat-lattice transformer[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Seattle: ACL, 2020: 6836-6842.
[16] 陈明, 刘蓉, 张晔. 基于多重注意力机制的中文医疗实体识别[J]. 计算机工程, 2023, 49(6): 314-320.
CHEN M, LIU R, ZHANG Y. Chinese medical entity recognition based on multiple attention mechanism [J]. Computer Engineering, 2023, 49(6): 314-320.
[17] 梁倬骞, 王东, 朱慧, 等. 基于领域本体的网络财务报告文本信息抽取研究[J]. 广东工业大学学报, 2017, 34(3): 89-95.
LIANG Z Q, WANG D, ZHU H, et al. A research on text information extraction from annual report based on domain ontology [J]. Journal of Guangdong University of Technology, 2017, 34(3): 89-95.
[18] LIU S Y, DUAN J W, GONG F, et al. Fusing label relations for Chinese EMR named entity recognition with machine reading comprehension[C]//18th International Symposium on Bioinformatics Research and Applications. Berlin: Springer, 2023: 41-51.
[19] 赵萍, 窦全胜, 唐焕玲, 等. 融合词信息嵌入的注意力自适应命名实体识别[J]. 计算机工程与应用, 2023(8): 167-174.
ZHAO P, DOU Q S, TANG H L, et al. Attention adaptive model with word information embedding for named entity recognition [J]. Computer Engineering and Applications, 2023(8): 167-174.
[20] WU S, SONG X N, FENG Z H. MECT: multi-metadata embedding based cross-transformer for Chinese named entity recognition[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Bangkok: ACL, 2021: 1529-1539.
[21] XUE M G, YU B W, LIU T W, et al. Porous lattice transformer encoder for Chinese NER[C]//Proceedings of the 28th International Conference on Computational Linguistics. Barcelona: ICCL, 2020: 3831-3841.
[22] MA R T, PENG M L, ZHANG Q, et al. Simplify the usage of lexicon in Chinese NER[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Seattle: ACL, 2020: 5951-5960.
[1] Li Zhuo-zhang, Xu Bo-yan, Cai Rui-chu, Hao Zhi-feng. Speaker-Aware Cross Attention Speaker Extraction Network [J]. Journal of Guangdong University of Technology, 2024, 41(03): 91-101.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © 2017 Journal of Guangdong University of Technology
Add：729 East Dongfeng RD., Guangzhou PRC. Postcode: 510090
Tel：020-37626653,020-37626139,020-37626396
Email：xbzrb@gdut.edu.cn
Supported:Beijing Magtech