Journal of Guangdong University of Technology ›› 2023, Vol. 40 ›› Issue (04): 31-36.doi: 10.12052/gdutxb.220111

• Computer Science and Technology • Previous Articles     Next Articles

Research on Inductive Transfer Learning Model Based on Self-paced Learning Strategy

Zhang Yu1, Liu Bo2   

  1. 1. School of Computer Science and Technology, Guangdong University of Technology, Guangzhou 510006, China;
    2. School of Automation, Guangdong University of Technology, Guangzhou 510006, China
  • Received:2022-06-24 Online:2023-07-25 Published:2023-08-02

Abstract: Machine learning tasks are usually single-task learning. However, in practical applications, the learning tasks are often related. As a result, the correlation between tasks is often ignored, and the complexity of samples is sometimes not considered in single-task learning. To address this, this paper proposes a new inductive transfer learning model based on the self-paced learning strategyby constructing a prediction model to learn the shared parameters of multiple related source tasks. First, the proposed model uses the strategy of self-paced learning to jointly learns the multiple related source tasks, and weights the learning samples according to the loss and difficulty degree of the samples in the source tasks. The parameters of the self-paced learning are iteratively updated to select the optimal samples with the less loss. Then, the knowledge learned from multiple related source tasks guide the learning of target tasks to construct multiple models for related transfer learning target tasks, and transfer these models to related target tasks to improve their generalization ability. Finally, we optimize the target model by using the Lagrange function to improve the performance of the classifiers. Experimental results show that the proposed model is superior to the existing transfer learning model under the same experimental conditions.

Key words: self-paced learning, inductive transfer learning, support vector machines

CLC Number: 

  • TP301
[1] BENGIO Y, LOURADOUR J, COLLOBERT R, et al. Curriculum learning[C]//Proceedings of the 26th Annual International Conference on Machine Learning. Montreal: ACM, 2009, 382: 41-48.
[2] KUMAR M, PACKER B, KOLLER D. Self-paced learning for latent variable models[C]//Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010. Vancouver: Curran Associates, Inc., 2010: 1189-1197.
[3] BENGIO Y, COURVILLE A C, VINCENT P. Representation learning: a review and new perspectives[J]. Journal of Electronic Imaging, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(8): 1798-1828.
[4] WANG Y, GAN W, YANG J, et al. Dynamic curriculum learning for imbalanced data classification[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 5017-5026.
[5] SOVIANY P, TUDOR R, IONESCU. Curriculum learning: a survey[J]. International Journal of Computer Vision, 2022, 130(6): 1526-1565.
[6] YIN T, LIU N, SUN H. Self-paced active learning for deep CNNs via effective loss function[J]. Neurocomputing, 2021, 424: 1-8.
[7] YU H, WEN G, GAN J. Self-paced learning for K-means clustering algorithm[J]. Pattern Recognition Letters, 2020, 132: 69-75.
[8] XIAO Y, YANG X, LIU B. A new self-paced method for multiple instance boosting learning[J]. Information Sciences, 2020, 515: 80-90.
[9] PAN S J, YANG Q. A Survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359.
[10] CRAWFORD M, KHOSHGOFTAAR T M. Using inductive transfer learning to improve hotel review spam detection[C]//2021 IEEE 22nd International Conference on Information Reuse and Integration for Data Science (IRI). Las Vegas, NV: IEEE, 2021: 248-254.
[11] YUN H, ZHANG C, HOU H, et al. An adaptive approach for ice detection in wind turbine with inductive transfer learning[J]. IEEE Access, 2019, 7: 122205-122213.
[12] DENG Z, JIANG Y, WANG S, et al. Enhanced knowledge leverage-based TSK fuzzy system modeling for inductive transfer learning[J]. ACM Transactions on Intelligent Systems and Technology, 2016, 8(1): 11: 1-11: 21.
[13] 黎启祥, 肖燕珊, 郝志峰, 等. 基于抗噪声的多任务多示例学习算法研究[J]. 广东工业大学学报, 2018, 35(3): 47-53.LI Q X, XIAO Y S, HAO Z F, et al. An algorithm based on multi-task multi-instance anti-noise learning.[J]. Journal of Guangdong University of Technology, 2018, 35(3): 47-53.
[14] XIAO Y, FENG J, LIU B. A new transductive learning method with universum data[J]. Applied Intelligence, 2021, 51(8): 5571-5583.
[15] XIAO F, PANG L, LAN Y, et al. Transductive learning for unsupervised text style transfer[C]// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Punta Cana: Association for Computational Linguistics, 2021, 2510-2521.
[16] ZHAO P, WU G, YAO S, et al. A transductive transfer learning approach based on manifold learning[J]. Computing in Science and Engineering, 2020, 22(1): 77-87.
[17] MICHAU G, FINK O. Unsupervised transfer learning for anomaly detection: application to complementary operating condition transfer[J]. Knowledge-Based Systems, 2021, 216: 106816.
[18] CHEN X, YANG R, WEN H. Transfer learning with unsupervised domain adaptation method for bearing fault diagnosis[C]//2021 CAA Symposium on Fault Detection, Supervision, and Safety for Technical Processes (SAFEPROCESS). Chengdu: IEEE, 2021: 1-6.
[19] ROCHA G, CARDOSO H L. Improving transfer learning in unsupervised language adaptation[C]//International Conference on Artificial Neural Networks. Bratislava: Springer, 2021, 12895: 588-599.
[20] MEI B, XU Y. Multi-task least squares twin support vector machine for classification[J]. Neurocomputing, 2019, 338: 26-33.
[1] Wang Dan-rong, Mo Yan. Support Vector Machines Based Method to Solve Discrete Linear Differential Equations [J]. Journal of Guangdong University of Technology, 2020, 37(02): 87-93.
[2] ZHU Yan-fei1,TAN Hong-zhou2,ZHANG Yun1. Blind Nonlinear System Identification Based on LS-SVM [J]. Journal of Guangdong University of Technology, 2007, 24(2): 76-79.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!