广东工业大学学报 ›› 2025, Vol. 42 ›› Issue (01): 33-41.doi: 10.12052/gdutxb.240017

• 智慧医疗 • 上一篇    下一篇

基于多尺度多实例学习的非小细胞肺癌亚型分类方法

罗超繁1, 刘震宇2   

  1. 1. 广东工业大学 计算机学院, 广东 广州 510006;
    2. 广东工业大学 信息工程学院, 广东 广州 510006
  • 收稿日期:2024-01-30 出版日期:2025-01-25 发布日期:2025-01-14
  • 通信作者: 刘震宇(1976–)，男，副研究员，博士，主要研究方向为深度学习、图像处理等，E-mail：zhenyuliu@gdut.edu.cn
  • 作者简介:罗超繁(1999–)，男，硕士研究生，主要研究方向为深度学习、医学图像处理等，E-mail：2112105228@mail2.gdut.edu.cn
  • 基金资助:
    2021年广东省企业科技特派员专项项目(GDKTP2021011000)

Non-small Cell Lung Cancer Subtype Classification Method Based on Multi-scale Multi-instance Learning

Luo Chaofan1, Liu Zhenyu2   

  1. 1. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China;
    2. School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China
  • Received:2024-01-30 Online:2025-01-25 Published:2025-01-14

HTML

49

PDF (PC)

106

摘要： 非小细胞肺癌的准确诊断与亚型鉴别对指导患者的个体化精准治疗意义重大。但由于非小细胞肺癌固有的肿瘤异质性，导致其同一亚型的病理形态差异较大,而不同亚型之间也可能存在一定的形态学重叠，给临床诊断带来很大困难。为解决这一问题，本文设计了一种基于多实例学习的多尺度特征提取与融合的计算机辅助诊断框架。首先，该框架在细胞级、组织级等不同层次对病理图像进行多尺度采样和特征提取。然后，利用Transformer网络实现端到端的多尺度信息融合，建模不同粒度实例之间的依赖关系。最后，本文设计了一种基于注意力的实例损失，通过区分最具区分性实例的特征，提供了额外的监督信号，以进一步提高模型的分类性能。在包含1 674张病理切片的公开数据集上的实验结果显示，本文方法可以更充分地利用病理图像的多粒度信息，显著提高非小细胞肺癌亚型的分类准确率。并且所提方法的注意力热图具有良好的可解释性，可以直观判断单个样本的分类质量，为后续模型优化提供了定量分析方法。

关键词: 非小细胞肺癌, 组织病理图像, 多实例学习, Transformer, 多尺度特征融合

Abstract: Accurate diagnosis and subtyping of non-small cell lung cancer (NSCLC) are crucial for providing patient-specific precision treatment. However, the inherent tumor heterogeneity of NSCLC leads to significant morphological variations within the same subtype and similarities across different subtypes, presenting substantial challenges for pathologists. To address this issue, this study proposes a novel computer-aided diagnostic framework that integrates multi-scale feature extraction and fusion through multi-instance deep learning. The proposed method aims to effectively leverage the heterogeneous information presented in pathological whole-slide images (WSIs) to improve the accuracy of NSCLC subtype classification. Initially, the framework performs multi-scale sampling and feature extraction from WSIs at various levels, such as cellular and tissue levels, to capture both local and global contextual information. Subsequently, a vision transformer network is employed to model the complex dependencies among instances of varying granularity, facilitating end-to-end fusion of the extracted features for accurate classification. Furthermore, we introduce an attention-based instance loss function that adaptively weighs the contribution of each instance based on its discriminative power, providing additional supervision to enhance the classification performance of the model. We evaluat our method on a large public dataset containing 1 674 H&E-stained pathological slide images of NSCLC. The experimental results demonstrate that our multi-scale fusion method effectively leverages the rich information in multi-grained pathological data, significantly outperforming single-scale approaches in NSCLC subtype classification accuracy. Moreover, the method's attention heatmaps offer interpretability and allow for intuitive assessment of individual sample classification quality, serving as a quantitative analytical tool for further model refinement and validation. In conclusion, the proposed multi-scale multi-instance learning framework provides a powerful and interpretable solution for accurate NSCLC subtype classification, which has the potential to assist pathologists in making more reliable diagnostic decisions and ultimately improve patient care.

Key words: non-small cell lung cancer, histopathological images, multiple instance learning, Transformer, multi-scale feature fusion

中图分类号: 

  • TP183
[1] ZHENG R, ZHANG S, ZENG H, et al. Cancer incidence and mortality in China, 2016[J]. Journal of the National Cancer Center, 2022, 2(1): 1-9.
[2] SUNG H, FERLAY J, SIEGEL R L, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J]. CA: A Cancer Journal for Clinicians, 2021, 71(3): 209-249.
[3] COUDRAY N, OCAMPO P S, SAKELLAROPOULOS T, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning[J]. Nature Medicine, 2018, 24(10): 1559-1567.
[4] ZHAO L, XU X, HOU R, et al. Lung cancer subtype classification using histopathological images based on weakly supervised multi-instance learning[J]. Physics in Medicine & Biology, 2021, 66(23): 235013.
[5] WANG X, CHEN H, GAN C, et al. Weakly supervised deep learning for whole slide lung cancer image analysis[J]. IEEE Transactions on Cybernetics, 2019, 50(9): 3950-3962.
[6] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//31st Conference on Neural Information Processing Systems. Long Beach: MIT Press, 2017: 5998-6008.
[7] LI B, LI Y, ELICEIRI K W. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 14318-14328.
[8] DING S, WANG J, LI J, et al. Multi-scale prototypical transformer for whole slide image classification[C]//GREENSPAN H, MADABHUSHI A, MOUSAVI P, et al. Medical Image Computing and Computer Assisted Intervention-MICCAI 2023. Cham: Springer Nature Switzerland, 2023: 602-611.
[9] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: transformers for image recognition at scale[EB/OL]. arXiv: 2010.11929(2021-06-03) [2024-04-10]. https://doi.org/10.48550/arXiv.2010.11929.
[10] CAO L, WANG J, ZHANG Y, et al. E2EFP-MIL: end-to-end and high-generalizability weakly supervised deep convolutional network for lung cancer classification from whole slide image[J]. Medical Image Analysis, 2023, 88: 102837.
[11] 叶紫璇, 肖满生, 肖哲. 基于EfficientNet模型的多特征融合肺癌病理图像分型[J]. 湖南工业大学学报, 2021, 35(2): 51-57.
YE Z X, XIAO M S, XIAO Z. Lung cancer pathological image classification based on an efficientnet model with multi-feature fusion[J]. Journal of Hunan University of Technology, 2021, 35(2): 51-57.
[12] YU K H, ZHANG C, BERRY G J, et al. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features[J]. Nature Communications, 2016, 7(1): 12474.
[13] 朱滋陵. 基于细胞病理图像的肺癌亚型分类方法研究[D]. 沈阳: 沈阳工业大学, 2023.
[14] ILSE M, TOMCZAK J, WELLING M. Attention-based deep multiple instance learning[C]// Proceedings of the 35th International Conference on Machine Learning. Stockholm: PMLR, 2018: 2127-2136.
[15] LU M Y, WILLIAMSON D F, CHEN T Y, et al. Data-efficient and weakly supervised computational pathology on whole-slide images[J]. Nature Biomedical Engineering, 2021, 5(6): 555-570.
[16] CAMPANELLA G, HANNA M G, GENESLAW L, et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images[J]. Nature Medicine, 2019, 25(8): 1301-1309.
[17] SHAO Z, BIAN H, CHEN Y, et al. Transmil: transformer based correlated multiple instance learning for whole slide image classification[J]. Advances in Neural Information Processing Systems, 2021, 34: 2136-2147.
[18] SHI J, TANG L, GAO Z, et al. MG-Trans: multi-scale graph transformer with information bottleneck for whole slide image classification[J]. IEEE Transactions on Medical Imaging, 2023, 42(12): 3871-3883.
[19] OTSU N. A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1): 62-66.
[20] DENG J, DONG W, SOCHER R, et al. Imagenet: a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 248-255.
[21] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[22] NAGRANI A, YANG S, ARNAB A, et al. Attention bottlenecks for multimodal fusion[J]. Advances in Neural Information Processing Systems, 2021, 34: 14200-14213.
[23] CHOROMANSKI K, LIKHOSHERSTOV V, DOHAN D, et al. Rethinking attention with performers[EB/OL]. arXiv: 2009.14794(2012-11-19) [2024-04-10]. https://doi.org/10.48550/arXiv.2009.14794.
[24] KINGMA D P, BA J. Adam: a method for Stochastic Optimization[EB/OL]. arXiv: 1412.6980(2017-01-30) [2024-04-10]. https://doi.org/10.48550/arXiv.1412.6980.
[1] 曾安, 王丹, 杨宝瑶, 张小波, 石镇维, 刘再毅, 潘丹. 基于Transformer与注意力机制的肺部肿瘤分割方法[J]. 广东工业大学学报, 2025, 42(01): 24-32.
[2] 冯广, 鲍龙. 基于红外可见光融合的复杂环境下人脸识别方法[J]. 广东工业大学学报, 2024, 41(03): 62-70,109.
[3] 郭傲, 许柏炎, 蔡瑞初, 郝志峰. 基于时序对齐的风格控制语音合成算法[J]. 广东工业大学学报, 2024, 41(02): 84-92.
[4] 赖志茂, 章云, 李东. 基于Transformer的人脸深度伪造检测技术综述[J]. 广东工业大学学报, 2023, 40(06): 155-167.
[5] 赵智丽， 韩雅莉. ERα通过Notch1信号通路刺激NCI-H23细胞增殖[J]. 广东工业大学学报, 2016, 33(03): 88-92.
Viewed
Full text
106
HTML PDF
Just accepted Online first Issue Just accepted Online first Issue
0 0 0 0 4 102

  From Others local
  Times 35 71
  Rate 33% 67%

Abstract
121
Just accepted Online first Issue
0 11 110
  From local
  Times 121
  Rate 100%

Cited
Web of Science  Crossref   ScienceDirect  Search for Citations in Google Scholar >>
 
This page requires you have already subscribed to WoS.
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2017 《广东工业大学学报》编辑部
地址：广州市东风东路729号广东工业大学 邮编：510090
电话：020-37626653,020-37626139,020-37626396
邮箱：xbzrb@gdut.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发