广东工业大学学报 ›› 2023, Vol. 40 ›› Issue (06): 176-184.doi: 10.12052/gdutxb.230065

• 人工智能 • 上一篇    

基于门控注意力单元的中文医学命名实体识别

吴晓鸰, 陈祥旺, 占文韬, 凌捷   

  1. 广东工业大学 计算机学院, 广东 广州 510006
  • 收稿日期:2023-05-06 出版日期:2023-11-25 发布日期:2023-11-08
  • 作者简介:吴晓鸰(1979-),女,副教授,博士,主要研究方向为人工智能、物联网,E-mail:xl.wu@gdut.edu.cn
  • 基金资助:
    广东省重点研发计划项目(2019B010139002) ；广东省国际科技合作项目(2019A050513010)

Chinese Medical Named Entity Recognition Based on Gated Attention Unit

Wu Xiao-ling, Chen Xiang-wang, Zhan Wen-tao, Ling Jie   

  1. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China
  • Received:2023-05-06 Online:2023-11-25 Published:2023-11-08

HTML

PDF (PC)

612

摘要： 医学命名实体识别任务是对电子病历中的医学实体进行自动识别和分类，对于下游任务例如信息检索、知识图谱等有着十分重要的作用。现有的方法忽略了实体间的依赖性，因此，本文提出了一种基于门控注意力单元的模型，首先利用预训练模型MC-BERT捕捉上下文语境信息，再利用交叉注意力和门控注意力单元提高实体查询和上下文语义之间的交互性，并提取实体间的依赖关系和关联性，最后，利用二分图的匹配算法，计算模型训练中的损失。本文在CMeEE、CMQNN和MSRA数据集上进行了实验，实验结果表明本文模型在3个数据集上的F1值分别达到了70.74%，96.92%和95.53%，优于其他相关模型，证明了本文模型在中文医学命名实体识别任务上的有效性。

关键词: 中文命名实体识别, 门控注意力单元, 交叉注意力, MC-BERT

Abstract: The medical named entity recognition aims to automatically identify and classify medical entities in electronic medical records, which plays a very important role in downstream tasks such as information retrieval and knowledge graph. Existing methods usually ignore the dependencies between entities. To address this, this paper proposes a gated attention unit-based model for Chinese medical named entity recognition. First, the proposed model uses the pre-training model MC-BERT to capture contextual information. Then, it uses the cross-attention and gated attention unit to enhance the interaction between entity query and contextual semantics, and further extract the dependency and correlation between entities. Finally, the proposed model uses the matching algorithm of bipartite graph to calculate the loss. This paper conducted experiments on three datasets, including the CMeEE, CMQNN, and MSRA. The experimental results show that the F1 values of the proposed model on the three datasets are 70.74%, 96.92%, and 95.53%, respectively, which outperforms other related models, demonstrating the effectiveness of the proposed model in Chinese medical named entity recognition task.

Key words: Chinese named entity recognition, gated attention unit, cross attention, MC-BERT

中图分类号: 

  • TP391.41
[1] 苏攀予, 黄健文. 基于知识图谱的BIM技术在工程造价领域的应用研究[J]. 广东工业大学学报, 2023, 40(2): 103-110.
SU P Y, HUANG J W. The application of BIM technology in the field of engineering cost based on knowledge graph [J]. Journal of Guangdong University of Technology, 2023, 40(2): 103-110.
[2] 曹智雄, 吴晓鸰, 骆晓伟, 等. 融合迁移学习与YOLOv5的安全帽佩戴检测算法[J]. 广东工业大学学报, 2023, 40(4): 67-76.
CAO Z X, WU X L, LUO X W, et al. Helmet wearing detection algorithm integrating transfer learning and YOLOv5 [J]. Journal of Guangdong University of Technology, 2023, 40(4): 67-76.
[3] WU S H, SHEN Y L, TAN Z Q, et al. Propose-and-refine: a two-stage set prediction network for nested named entity recognition[C]//Proceedings of the 31th International Joint Conference on Artificial Intelligence. Montreal: IJCAI, 2022: 4418-4424.
[4] WANG B L, LU W. Combining spans into entities: a neural two-stage approach for recognizing discontiguous entities[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the Ninth International Joint Conference on Natural Language Processing. Hong Kong: ACL, 2019: 6217-6225.
[5] LI F, LIN Z C, ZHANG M S, et al. A span-based model for joint overlapped and discontinuous named entity recognition[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Bangkok: ACL, 2021: 4814-4828.
[6] GU Y J, QU X Y, WANG Z F, et al. Delving deep into regularity: a simple but effective method for Chinese named entity recognition[C]//Findings of the Association for Computational Linguistics: NAACL 2022. Seattle: ACL, 2022: 1863-1873.
[7] LUAN Y, WADDEN D, HE L H, et al. A general framework for information extraction using dynamic span graphs[C]//Proceedings of the 2019 Conference of the North American Chapter of the ACL. Florence: ACL, 2019: 3036-3046.
[8] YU J T, BOHNET B, POESIO M. Named entity recognition as dependency parsing[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Seattle: ACL, 2020: 6470-6476.
[9] WANG Q, ZHOU Y M, RUAN T, et al. Incorporating dictionaries into deep neural networks for the Chinese clinical named entity recognition [J]. Journal of Biomedical Informatics, 2019, 92: 103133.
[10] LI X Y, ZHANG H, ZHOU X H. Chinese clinical named entity recognition with variant neural structures based on BERT methods [J]. Journal of Biomedical Informatics, 2020, 107: 103422.
[11] LI X Y, FENG J R, MENG Y X, et al. A unified MRC framework for named entity recognition[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Seattle: ACL, 2020: 5849-5859.
[12] PAN Y, XIN C, SUN Z Y, et al. Enhanced language representation with label knowledge for span extraction[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana: ACL, 2021: 4623-4635.
[13] HANG Y, GUI T, DAI J Q, et al. A unified generative framework for various NER subtasks[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Bangkok: ACL, 2021: 5808-5822.
[14] SHEN Y L, WANG X B, TAN Z Q, et al. Parallel instance query network for named entity recognition[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Dublin: ACL, 2022: 947-961.
[15] LI X N, YAN H, QIU X P, et al. FLAT: Chinese NER using flat-lattice transformer[C]// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Seattle: ACL, 2020: 6836-6842.
[16] 陈明, 刘蓉, 张晔. 基于多重注意力机制的中文医疗实体识别[J]. 计算机工程, 2023, 49(6): 314-320.
CHEN M, LIU R, ZHANG Y. Chinese medical entity recognition based on multiple attention mechanism [J]. Computer Engineering, 2023, 49(6): 314-320.
[17] 梁倬骞, 王东, 朱慧, 等. 基于领域本体的网络财务报告文本信息抽取研究[J]. 广东工业大学学报, 2017, 34(3): 89-95.
LIANG Z Q, WANG D, ZHU H, et al. A research on text information extraction from annual report based on domain ontology [J]. Journal of Guangdong University of Technology, 2017, 34(3): 89-95.
[18] LIU S Y, DUAN J W, GONG F, et al. Fusing label relations for Chinese EMR named entity recognition with machine reading comprehension[C]//18th International Symposium on Bioinformatics Research and Applications. Berlin: Springer, 2023: 41-51.
[19] 赵萍, 窦全胜, 唐焕玲, 等. 融合词信息嵌入的注意力自适应命名实体识别[J]. 计算机工程与应用, 2023(8): 167-174.
ZHAO P, DOU Q S, TANG H L, et al. Attention adaptive model with word information embedding for named entity recognition [J]. Computer Engineering and Applications, 2023(8): 167-174.
[20] WU S, SONG X N, FENG Z H. MECT: multi-metadata embedding based cross-transformer for Chinese named entity recognition[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Bangkok: ACL, 2021: 1529-1539.
[21] XUE M G, YU B W, LIU T W, et al. Porous lattice transformer encoder for Chinese NER[C]//Proceedings of the 28th International Conference on Computational Linguistics. Barcelona: ICCL, 2020: 3831-3841.
[22] MA R T, PENG M L, ZHANG Q, et al. Simplify the usage of lexicon in Chinese NER[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Seattle: ACL, 2020: 5951-5960.
[1] 李卓璋, 许柏炎, 蔡瑞初, 郝志峰. 说话人感知的交叉注意力说话人提取网络[J]. 广东工业大学学报, 2024, 41(03): 91-101.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2017 《广东工业大学学报》编辑部
地址：广州市东风东路729号广东工业大学 邮编：510090
电话：020-37626653,020-37626139,020-37626396
邮箱：xbzrb@gdut.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发