一种基于邻居环境感知的主动领域自适应算法

doi:10.12052/gdutxb.230172

摘要/Abstract

摘要： 主动领域自适应 (Active Domain Adaptation, ADA) 目的是在领域自适应的背景下利用尽可能少的目标域查询预算来训练一个有效模型。然而，由于领域漂移的存在，现有的算法选择的实例可能是信息量低、冗余或离群的。为了解决这个问题，本文提出了一种新的方法用于主动领域自适应，即基于邻居环境感知的样本选择算法 (Neighbor Environment Perception Sample Selection, NEPS) 。NEPS以一种邻居环境感知的方式来探索目标样本的信息量，以选择在领域转移下可能最有价值的实例。具体而言，在计算样本信息量时，测量所提出的邻居环境感知信息得分 (Neighbor Awareness Informativeness Score, NAIS) 的同时利用从单个数据点以及从其附近的邻居所获得的信息，从而确保所选样本具有较高的单点及环境信息量。同时，通过计算候选样本与已标记样本的相似度分数来对样本进行排序挑选，以保证所选样本的多样性。此外，还充分利用了所有已标记样本以及目标领域的大量未标记数据信息，进而提高模型的性能。经验证，本文方法具有较强的样本选择能力，在各种基准数据集上的分类效果都优于现有的模型。

关键词: 主动领域自适应, 信息性, 多样性, 邻居环境感知

Abstract: Active domain adaptation (ADA) aims to train an effective model under the context of domain adaptation with as few queried instances as possible. However, existing algorithms tend to select instances that are either uninformative, redundant, or outliers due to domain shift. To address this issue, a novel approach called neighbor environment perception sample selection (NEPS) for active domain adaptation is proposed. NEPS explores the target sample informativeness in a neighbor environment-aware manner to select instances that are potentially most valuable under domain shift. Specifically, from informativeness perspective, NEPS aims to acquire knowledge not only from individual data points but also from their neighboring samples. This is achieved by measuring neighbor awareness informativeness score (NAIS) , which ensures the selected samples have both high individual informativeness score and environment informativeness score. Additionally, NEPS ranks and selects samples based on their similarity scores with labeled samples to ensure diversity among the chosen instances. Furthermore, NEPS makes effective use of all labeled samples as well as a large amount of unlabeled data from the target domain to enhance the model's performance. Experimental results demonstrate that NEPS exhibits strong sample selection capability and outperforms existing models in terms of classification performance on various benchmark datasets.

Key words: active domain adaptation, informativeness, diversity, neighbor environment perception

中图分类号:

TP181

陈鑫瑀, 朱鉴, 陈炳丰, 蔡瑞初. 一种基于邻居环境感知的主动领域自适应算法[J]. 广东工业大学学报,doi: 10.12052/gdutxb.230172

Chen Xin-yu, Zhu Jian, Chen Bing-feng, Cai Rui-chu. Active Domain Adaptation Based on Neighbor Environment Perception Sample Selection[J]. Journal of Guangdong University of Technology, doi: 10.12052/gdutxb.230172

参考文献

[1] GANIN Y, USTINOVA E, AJAKAN H, et al. Domain-adversarial training of neural networks [J]. The Journal of Machine Learning Research, 2016, 17(1): 2096-2030.
[2] 杨国庆, 郭本华, 钱淑渠, 等. 基于伪标签的无监督领域自适应分类方法[J]. 计算机应用研究, 2022, 39(5): 1357-1361.
YANG G Q, GUO B H, QIAN S Q, et al. Pseudo label based unsupervised domain adaptation classification method [J]. Application Research of Computers, 2022, 39(5): 1357-1361.
[3] 李晶晶, 孟利超, 张可, 等. 领域自适应研究综述[J]. 计算机工程, 2021, 47(6): 1-13.
LI J J, MENG L C, ZHANG K, et al. Review of studies on domain adaptation [J]. Computer Engineering, 2021, 47(6): 1-13.
[4] CHEN X K, SHAO H D, XIAO Y M, et al. Collaborative fault diagnosis of rotating machinery via dual adversarial guided unsupervised multi-domain adaptation network [J]. Mechanical Systems and Signal Processing, 2023, 198: 110427.
[5] TZENG E, HOFFMAN J, ZHANG N, et al. Deep domain confusion: Maximizing for domain invariance [EB/OL]. arXiv: 1412.3474 (2014-12-10)[2024-03-13]. https://doi.org/10.48550/arXiv.1412.3474.
[6] HOFFMAN J, TZENG E, PARK T, et al. Cycada: cycle-consistent adversarial domain adaptation [C]//International Conference on Machine Learning (ICML) . Stockholm: ACM, 2018: 1989-1998.
[7] TSAI Y H, HUNG W C, SCHULTER S, et al. Learning to adapt structured output space for semantic segmentation [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Salt Lake City: IEEE, 2018: 7472-7481.
[8] ZHAO H, DES COMBES R T, ZHANG K, et al. On learning invariant representations for domain adaptation [C]//International Conference on Machine Learning (ICML) . California: ACM, 2019: 7523-7532.
[9] 包震伟, 刘丹, 米金鹏. 弱监督与少样本学习场景下视频行为识别综述[J]. 计算机应用研究, 2023, 40(6): 1629-1635.
BAO Z W, LIU D, MI J P. Review of video action recognition under weak supervision and few-shot learning [J]. Application Research of Computers, 2023, 40(6): 1629-1635.
[10] SAITO K, KIM D, SCLAROFF S, et al. Semi-supervised domain adaptation via minimax entropy [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) . Seoul: IEEE, 2019: 8050-8058.
[11] 潘雪玲, 李国和, 郑艺峰. 面向深度网络的小样本学习综述[J]. 计算机应用研究, 2023, 40(10): 2881-2888.
PAN X L, LI G H, ZHENG Y F. Survey on few-shot learning for deep network [J]. Application Research of Computers, 2023, 40(10): 2881-2888.
[12] TESHIMA T, SATO I, SUGIYAMA M. Few-shot domain adaptation by causal mechanism transfer [C]//International Conference on Machine Learning (ICML) . Vienna: ACM, 2020: 9458-9469.
[13] 王月, 杨春宇, 郭鑫平, 等. 深度子领域迁移学习: 一种细粒度主动迁移视角[J]. 南京理工大学学报, 2023, 47(1): 90-102.
WANG Y, YANG C Y, GUO X P, et al. Deep subdomain transfer learning: a fine-grained active transfer perspective [J]. Journal of Nanjing University of Science And Technology, 2023, 47(1): 90-102.
[14] WANG D, SHANG Y. A new active labeling method for deep learning [C]//International Joint Conference on Neural Networks (IJCNN) . Beijing: IEEE, 2014: 112-119.
[15] LEWIS D D, CATLETT J. Heterogeneous uncertainty sampling for supervised learning [M]. Machine learning proceedings 1994. New Jersey: Morgan Kaufmann, 1994: 148-156.
[16] GAL Y, ISLAM R, GHAHRAMANI Z. Deep bayesian active learning with image data [C]//International Conference on Machine Learning (ICML) . Sydney: ACM, 2017: 1183-1192.
[17] SENER O, SAVARESE S. Active learning for convolutional neural networks: a core-set approach [EB/OL]. arXiv: 1708.00489 (2018-06-01)[2024-03-13]. https://doi.org/10.48550/arXiv.1708.00489.
[18] ZHDANOV F. Diverse mini-batch active learning [EB/OL]. arXiv: 1901.05954 (2019-01-17)[2024-03-13]. https://doi.org/10.48550/arXiv.1901.05954.
[19] ASH J T, ZHANG C, KRISHNAMURTHY A, et al. Deep batch active learning by diverse, uncertain gradient lower bounds [EB/OL]. arXiv: 1906.03671 (2020-02-24)[2024-03-13].https://doi.org/10.48550/arXiv.1906.03671.
[20] SHUI C J, ZHOU F, GAGNE C, et al. Deep active learning: unified and principled method for query and training [C]//International Conference on Artificial Intelligence and Statistics (ICAIS) . Hohhot: LNCS, 2020: 1308-1318.
[21] RAI P, SAHA A, DAUME III H, et al. Domain adaptation meets active learning [C]//Proceedings of the NAACL HLT 2010 Workshop on Active Learning for Natural Language Processing. California: ACL, 2010: 27-32.
[22] PRABHU V, CHANDRASEKARAN A, SAENKO K, et al. Active domain adaptation via clustering uncertainty-weighted embeddings [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) . Montreal: IEEE, 2021: 8505-8514.
[23] FU B, CAO Z J, WANG J M, et al. Transferable query selection for active domain adaptation [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Kuala Lumpur: IEEE, 2021: 7272-7281.
[24] XIE B H, YUAN L H, LI S, et al. Active learning for domain adaptation: an energy-based approach [C]//Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: AAAI, 2022: 8708-8716.
[25] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets [C]//Advances in Neural Information Processing Systems (NIPS) . Montreal: MIT Press, 2014.
[26] VENKATESWARA H, EUSEBIO J, CHAKRABORTY S, et al. Deep hashing network for unsupervised domain adaptation [C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Hawaii: IEEE, 2017: 5018-5027.
[27] SAENKO K, KULIS B, FRITZ M, et al. Adapting visual category models to new domains [C]//European Conference on Computer Vision (ECCV) . Hersonissos: Springer, 2010, 213-226.
[28] The imageclef-da challenge 2014 [EB/OL]. (2014-09-18)[2024-03-13].https://www.imageclef.org/2014/.
[29] KRIZHEVSKY A, HINTON G. Learning multiple layers of features fromtiny images[EB/OL]. (2009-04-08) [2024-03-13]. http://www.cs.utoronto.ca/~kriz/learning-features-2009-TR.
[30] SU J C, TSAI Y H, SOHN K, et al. Active adversarial domain adaptation [C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) . Colorado: IEEE, 2020: 739-748.
[31] ZHOU F, SHUI C J, YANG S C, et al. Discriminative active learning for domain adaptation [J]. Knowledge Based Systems, 2021, 222: 106986.
[32] DE MATHELIN A, DEHEEGER F, MOUGEOT M, et al. Discrepancy-based active learning for domain adaptation [EB/OL]. arXiv: 2103.03757 (2022-09-14)[2024-03-13].https://doi.org/10.48550/arXiv.2103.03757.
[33] RANGWANI H, JAIN A, AITHAL S K, et al. S³VAADA: submodular subset selection for virtual adversarial active domain adaptation [C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) . Montreal: IEEE, 2021: 7516-7525.
[34] HAN K, KIM Y, HAN D, et al. TL-ADA: transferable Loss-based active domain adaptation [J]. Neural Networks, 2023, 161: 670-681.
[35] JOSHI A J, PORIKLI F, PAPANIKOLOPOULOS N. Multi-class active learning for image classification [C]//2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Miami: IEEE, 2009: 2372-2379.
[36] AGARWAL S, ARORA H, ANAND S, et al. Contextual diversity for active learning [C]//European Conference on Computer Vision (ECCV) . Glasgow: Springer, 2020: 137-153.
[37] VAN DER MAATEN L, HINTON G. Visualizing data using tsne [J]. Journal of Machine Learning Research, 2008, 9(11): 2579-2605.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed