广东工业大学学报 ›› 2023, Vol. 40 ›› Issue (04): 31-36.doi: 10.12052/gdutxb.220111

• 计算机科学与技术 • 上一篇    下一篇

基于自步学习策略的归纳式迁移学习模型研究

张宇1, 刘波2   

  1. 1. 广东工业大学 计算机学院, 广东 广州 510006;
    2. 广东工业大学 自动化学院, 广东 广州 510006
  • 收稿日期:2022-06-24 出版日期:2023-07-25 发布日期:2023-08-02
  • 通信作者: 刘波(1978–)， 男，教授，博士，主要研究方向为机器学习、数据挖掘，E-mail：csboliu@163.com
  • 作者简介:张宇(1997–)， 男，硕士研究生，主要研究方向为机器学习、多任务学习、自步学习
  • 基金资助:
    国家自然科学基金资助项目(61876044，62076074)

Research on Inductive Transfer Learning Model Based on Self-paced Learning Strategy

Zhang Yu1, Liu Bo2   

  1. 1. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China;
    2. School of Automation, Guangdong University of Technology, Guangzhou 510006, China
  • Received:2022-06-24 Online:2023-07-25 Published:2023-08-02

HTML

PDF (PC)

705

摘要： 在机器学习任务中很多时候是单任务学习，所以往往会忽略学习任务之间的相关性，并且在单任务学习中容易忽略样本的复杂度，为此，本文提出了一种新的基于自步学习策略的归纳式迁移学习模型，通过对当前多个相关的源任务共享参数学习构建一个预测模型，从而解决目标任务的分类问题。首先提出模型基于自步学习的策略，按照预先设定的自步学习模型参数对多个相关的源任务进行联合学习，利用源任务中样本的损失大小与难易程度对学习的样本赋予一个权重，在迭代的过程中更新自步学习的参数从而挑选出比较合适的样本(损失较小的样本)，然后使用在多个相关的源任务中学习到的知识帮助学习目标任务，构建多个相关迁移学习目标任务的模型，将多个源任务学习到的模型迁移到相关的目标任务中从而提高模型的泛化能力，最后通过拉格朗日函数进一步优化目标模型以提高分类器的性能。实验结果表明，提出的模型在相同的实验条件下优于现有的归纳式迁移学习模型。

关键词: 自步学习, 归纳式迁移学习, 支持向量机

Abstract: Machine learning tasks are usually single-task learning. However, in practical applications, the learning tasks are often related. As a result, the correlation between tasks is often ignored, and the complexity of samples is sometimes not considered in single-task learning. To address this, this paper proposes a new inductive transfer learning model based on the self-paced learning strategyby constructing a prediction model to learn the shared parameters of multiple related source tasks. First, the proposed model uses the strategy of self-paced learning to jointly learns the multiple related source tasks, and weights the learning samples according to the loss and difficulty degree of the samples in the source tasks. The parameters of the self-paced learning are iteratively updated to select the optimal samples with the less loss. Then, the knowledge learned from multiple related source tasks guide the learning of target tasks to construct multiple models for related transfer learning target tasks, and transfer these models to related target tasks to improve their generalization ability. Finally, we optimize the target model by using the Lagrange function to improve the performance of the classifiers. Experimental results show that the proposed model is superior to the existing transfer learning model under the same experimental conditions.

Key words: self-paced learning, inductive transfer learning, support vector machines

中图分类号: 

  • TP301
[1] BENGIO Y, LOURADOUR J, COLLOBERT R, et al. Curriculum learning[C]//Proceedings of the 26th Annual International Conference on Machine Learning. Montreal: ACM, 2009, 382: 41-48.
[2] KUMAR M, PACKER B, KOLLER D. Self-paced learning for latent variable models[C]//Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010. Vancouver: Curran Associates, Inc., 2010: 1189-1197.
[3] BENGIO Y, COURVILLE A C, VINCENT P. Representation learning: a review and new perspectives[J]. Journal of Electronic Imaging, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8): 1798-1828.
[4] WANG Y, GAN W, YANG J, et al. Dynamic curriculum learning for imbalanced data classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 5017-5026.
[5] SOVIANY P, TUDOR R, IONESCU. Curriculum learning: a survey[J]. International Journal of Computer Vision, 2022, 130(6): 1526-1565.
[6] YIN T, LIU N, SUN H. Self-paced active learning for deep CNNs via effective loss function[J]. Neurocomputing, 2021, 424: 1-8.
[7] YU H, WEN G, GAN J. Self-paced learning for K-means clustering algorithm[J]. Pattern Recognition Letters, 2020, 132: 69-75.
[8] XIAO Y, YANG X, LIU B. A new self-paced method for multiple instance boosting learning[J]. Information Sciences, 2020, 515: 80-90.
[9] PAN S J, YANG Q. A Survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.
[10] CRAWFORD M, KHOSHGOFTAAR T M. Using inductive transfer learning to improve hotel review spam detection[C]//2021 IEEE 22nd International Conference on Information Reuse and Integration for Data Science (IRI). Las Vegas, NV: IEEE, 2021: 248-254.
[11] YUN H, ZHANG C, HOU H, et al. An adaptive approach for ice detection in wind turbine with inductive transfer learning[J]. IEEE Access, 2019, 7: 122205-122213.
[12] DENG Z, JIANG Y, WANG S, et al. Enhanced knowledge leverage-based TSK fuzzy system modeling for inductive transfer learning[J]. ACM Transactions on Intelligent Systems and Technology, 2016, 8(1): 11: 1-11: 21.
[13] 黎启祥, 肖燕珊, 郝志峰, 等. 基于抗噪声的多任务多示例学习算法研究[J]. 广东工业大学学报, 2018, 35(3): 47-53.LI Q X, XIAO Y S, HAO Z F, et al. An algorithm based on multi-task multi-instance anti-noise learning.[J]. Journal of Guangdong University of Technology, 2018, 35(3): 47-53.
[14] XIAO Y, FENG J, LIU B. A new transductive learning method with universum data[J]. Applied Intelligence, 2021, 51(8): 5571-5583.
[15] XIAO F, PANG L, LAN Y, et al. Transductive learning for unsupervised text style transfer[C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana: Association for Computational Linguistics, 2021, 2510-2521.
[16] ZHAO P, WU G, YAO S, et al. A transductive transfer learning approach based on manifold learning[J]. Computing in Science and Engineering, 2020, 22(1): 77-87.
[17] MICHAU G, FINK O. Unsupervised transfer learning for anomaly detection: application to complementary operating condition transfer[J]. Knowledge-Based Systems, 2021, 216: 106816.
[18] CHEN X, YANG R, WEN H. Transfer learning with unsupervised domain adaptation method for bearing fault diagnosis[C]//2021 CAA Symposium on Fault Detection, Supervision, and Safety for Technical Processes (SAFEPROCESS). Chengdu: IEEE, 2021: 1-6.
[19] ROCHA G, CARDOSO H L. Improving transfer learning in unsupervised language adaptation[C]//International Conference on Artificial Neural Networks. Bratislava: Springer, 2021, 12895: 588-599.
[20] MEI B, XU Y. Multi-task least squares twin support vector machine for classification[J]. Neurocomputing, 2019, 338: 26-33.
[1] 陈友鹏, 陈璟华. 基于鲸鱼优化参数的最小二乘支持向量机短期负荷预测方法[J]. 广东工业大学学报, 2020, 37(03): 75-81.
[2] 王丹荣, 莫艳. 基于支持向量机的离散线性微分方程求解方法[J]. 广东工业大学学报, 2020, 37(02): 87-93.
[3] 陈培文, 傅秀芬. 采用SVM方法的文本情感极性分类研究[J]. 广东工业大学学报, 2014, 31(3): 95-101.
[4] 胡俊, 滕少华, 张巍, 刘冬宁. 支持向量机与哈夫曼树实现多分类的研究[J]. 广东工业大学学报, 2014, 31(2): 36-42.
[5] 夏琴晔, 杨宜民. 基于biSCAN和SVM的机器人目标识别新算法研究[J]. 广东工业大学学报, 2013, 30(4): 65-69.
[6] 朱燕飞； 谭洪舟； 章云； . 基于LS-SVM的非线性系统盲辨识[J]. 广东工业大学学报, 2007, 24(2): 76-79.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 2017 《广东工业大学学报》编辑部
地址：广州市东风东路729号广东工业大学 邮编：510090
电话：020-37626653,020-37626139,020-37626396
邮箱：xbzrb@gdut.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发