Journal of Guangdong University of Technology ›› 2025, Vol. 42 ›› Issue (01): 33-41.doi: 10.12052/gdutxb.240017

• Smart Medical • Previous Articles     Next Articles

Non-small Cell Lung Cancer Subtype Classification Method Based on Multi-scale Multi-instance Learning

Luo Chaofan1, Liu Zhenyu2   

  1. 1. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China;
    2. School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China
  • Received:2024-01-30 Online:2025-01-25 Published:2025-01-14

Abstract: Accurate diagnosis and subtyping of non-small cell lung cancer (NSCLC) are crucial for providing patient-specific precision treatment. However, the inherent tumor heterogeneity of NSCLC leads to significant morphological variations within the same subtype and similarities across different subtypes, presenting substantial challenges for pathologists. To address this issue, this study proposes a novel computer-aided diagnostic framework that integrates multi-scale feature extraction and fusion through multi-instance deep learning. The proposed method aims to effectively leverage the heterogeneous information presented in pathological whole-slide images (WSIs) to improve the accuracy of NSCLC subtype classification. Initially, the framework performs multi-scale sampling and feature extraction from WSIs at various levels, such as cellular and tissue levels, to capture both local and global contextual information. Subsequently, a vision transformer network is employed to model the complex dependencies among instances of varying granularity, facilitating end-to-end fusion of the extracted features for accurate classification. Furthermore, we introduce an attention-based instance loss function that adaptively weighs the contribution of each instance based on its discriminative power, providing additional supervision to enhance the classification performance of the model. We evaluat our method on a large public dataset containing 1 674 H&E-stained pathological slide images of NSCLC. The experimental results demonstrate that our multi-scale fusion method effectively leverages the rich information in multi-grained pathological data, significantly outperforming single-scale approaches in NSCLC subtype classification accuracy. Moreover, the method's attention heatmaps offer interpretability and allow for intuitive assessment of individual sample classification quality, serving as a quantitative analytical tool for further model refinement and validation. In conclusion, the proposed multi-scale multi-instance learning framework provides a powerful and interpretable solution for accurate NSCLC subtype classification, which has the potential to assist pathologists in making more reliable diagnostic decisions and ultimately improve patient care.

Key words: non-small cell lung cancer, histopathological images, multiple instance learning, Transformer, multi-scale feature fusion

CLC Number: 

  • TP183
[1] ZHENG R, ZHANG S, ZENG H, et al. Cancer incidence and mortality in China, 2016[J]. Journal of the National Cancer Center, 2022, 2(1): 1-9.
[2] SUNG H, FERLAY J, SIEGEL R L, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J]. CA: A Cancer Journal for Clinicians, 2021, 71(3): 209-249.
[3] COUDRAY N, OCAMPO P S, SAKELLAROPOULOS T, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning[J]. Nature Medicine, 2018, 24(10): 1559-1567.
[4] ZHAO L, XU X, HOU R, et al. Lung cancer subtype classification using histopathological images based on weakly supervised multi-instance learning[J]. Physics in Medicine & Biology, 2021, 66(23): 235013.
[5] WANG X, CHEN H, GAN C, et al. Weakly supervised deep learning for whole slide lung cancer image analysis[J]. IEEE Transactions on Cybernetics, 2019, 50(9): 3950-3962.
[6] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//31st Conference on Neural Information Processing Systems. Long Beach: MIT Press, 2017: 5998-6008.
[7] LI B, LI Y, ELICEIRI K W. Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 14318-14328.
[8] DING S, WANG J, LI J, et al. Multi-scale prototypical transformer for whole slide image classification[C]//GREENSPAN H, MADABHUSHI A, MOUSAVI P, et al. Medical Image Computing and Computer Assisted Intervention-MICCAI 2023. Cham: Springer Nature Switzerland, 2023: 602-611.
[9] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: transformers for image recognition at scale[EB/OL]. arXiv: 2010.11929(2021-06-03) [2024-04-10]. https://doi.org/10.48550/arXiv.2010.11929.
[10] CAO L, WANG J, ZHANG Y, et al. E2EFP-MIL: end-to-end and high-generalizability weakly supervised deep convolutional network for lung cancer classification from whole slide image[J]. Medical Image Analysis, 2023, 88: 102837.
[11] 叶紫璇, 肖满生, 肖哲. 基于EfficientNet模型的多特征融合肺癌病理图像分型[J]. 湖南工业大学学报, 2021, 35(2): 51-57.
YE Z X, XIAO M S, XIAO Z. Lung cancer pathological image classification based on an efficientnet model with multi-feature fusion[J]. Journal of Hunan University of Technology, 2021, 35(2): 51-57.
[12] YU K H, ZHANG C, BERRY G J, et al. Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features[J]. Nature Communications, 2016, 7(1): 12474.
[13] 朱滋陵. 基于细胞病理图像的肺癌亚型分类方法研究[D]. 沈阳: 沈阳工业大学, 2023.
[14] ILSE M, TOMCZAK J, WELLING M. Attention-based deep multiple instance learning[C]// Proceedings of the 35th International Conference on Machine Learning. Stockholm: PMLR, 2018: 2127-2136.
[15] LU M Y, WILLIAMSON D F, CHEN T Y, et al. Data-efficient and weakly supervised computational pathology on whole-slide images[J]. Nature Biomedical Engineering, 2021, 5(6): 555-570.
[16] CAMPANELLA G, HANNA M G, GENESLAW L, et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images[J]. Nature Medicine, 2019, 25(8): 1301-1309.
[17] SHAO Z, BIAN H, CHEN Y, et al. Transmil: transformer based correlated multiple instance learning for whole slide image classification[J]. Advances in Neural Information Processing Systems, 2021, 34: 2136-2147.
[18] SHI J, TANG L, GAO Z, et al. MG-Trans: multi-scale graph transformer with information bottleneck for whole slide image classification[J]. IEEE Transactions on Medical Imaging, 2023, 42(12): 3871-3883.
[19] OTSU N. A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1): 62-66.
[20] DENG J, DONG W, SOCHER R, et al. Imagenet: a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 248-255.
[21] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[22] NAGRANI A, YANG S, ARNAB A, et al. Attention bottlenecks for multimodal fusion[J]. Advances in Neural Information Processing Systems, 2021, 34: 14200-14213.
[23] CHOROMANSKI K, LIKHOSHERSTOV V, DOHAN D, et al. Rethinking attention with performers[EB/OL]. arXiv: 2009.14794(2012-11-19) [2024-04-10]. https://doi.org/10.48550/arXiv.2009.14794.
[24] KINGMA D P, BA J. Adam: a method for Stochastic Optimization[EB/OL]. arXiv: 1412.6980(2017-01-30) [2024-04-10]. https://doi.org/10.48550/arXiv.1412.6980.
[1] Zeng An, Wang Dan, Yang Baoyao, Zhang Xiaobo, Shi Zhenwei, Liu Zaiyi, Pan Dan. Lung Tumor Segmentation Method Based on Transformer and Attention Mechanisms [J]. Journal of Guangdong University of Technology, 2025, 42(01): 24-32.doi: 10.12052/gdutxb.240017
[2] Zeng An, Pang Yao-xing, Pan Dan, Zhao Jing-liang. Segmentation of Left Ventricular Endocardium Using Direction-constrained Reinforcement Learning [J]. Journal of Guangdong University of Technology, 2024, 41(06): 60-68.doi: 10.12052/gdutxb.240017
[3] Chen Hong-qi, Luo De-xin, Lan Liang, Zhang Zhi-hao, Zhang Guo-hao. A Design of a 24-27 GHz Cascode High Gain Low Noise Amplifier [J]. Journal of Guangdong University of Technology, 2024, 41(06): 26-32.doi: 10.12052/gdutxb.240017
[4] Feng Guang, Bao Long. Face Recognition Method in Complex Environment Based on Infrared Visible Fusion [J]. Journal of Guangdong University of Technology, 2024, 41(03): 62-70,109.doi: 10.12052/gdutxb.240017
[5] Guo Ao, Xu Bo-yan, Cai Rui-chu, Hao Zhi-feng. Temporal Alignment Style Control in Text-to-Speech Synthesis Algorithm [J]. Journal of Guangdong University of Technology, 2024, 41(02): 84-92.doi: 10.12052/gdutxb.240017
[6] Lai Zhi-mao, Zhang Yun, Li Dong. A Survey of Deepfake Detection Techniques Based on Transformer [J]. Journal of Guangdong University of Technology, 2023, 40(06): 155-167.doi: 10.12052/gdutxb.240017
[7] Zhang Miao, Pang Zhuo-biao, Hao Xue-dong, Xie Si-wei, Zhang Xing-wang. A Research on a Transformerless Parallel Hybrid Active Power Filter [J]. Journal of Guangdong University of Technology, 2019, 36(05): 33-37.doi: 10.12052/gdutxb.240017
[8] Dong Wen-hua, Li Chun-lai, Lan Xiong. Design and Experimental Analysis of an Open-close Micro Current Transformer [J]. Journal of Guangdong University of Technology, 2019, 36(04): 65-69.doi: 10.12052/gdutxb.240017
[9] ZHAO Zhi-Li, HAN Ya-Li. Proliferation of NCI-H23 Promoted by ERα through Notch1 Signaling Pathway [J]. Journal of Guangdong University of Technology, 2016, 33(03): 88-92.doi: 10.12052/gdutxb.240017
[10] He Rui-wen, Xie Qiong-xiang, Cai Ze-xiang. Influence of Digital Acquisition of the Electrical Information on the Reliability of Relay Protection [J]. Journal of Guangdong University of Technology, 2013, 30(2): 68-73.doi: 10.12052/gdutxb.240017
[11] Chen He-en, , Feng Kai-ping, Pan Li-pei, Wu Yue-ming, . Study of Architecture Transformation [J]. Journal of Guangdong University of Technology, 2012, 29(2): 94-96.doi: 10.12052/gdutxb.240017
Viewed
Full text
112
HTML PDF
Just accepted Online first Issue Just accepted Online first Issue
0 0 0 0 4 108

  From Others local
  Times 38 74
  Rate 34% 66%

Abstract
139
Just accepted Online first Issue
0 11 128

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!