基于生成式样本合成的工件缺陷样本数据增强

摘要/Abstract

参考文献

Metrics

doi:10.12052/gdutxb.240056

摘要： 针对深度学习模型在工业缺陷视觉检测领域中因样本稀缺而难以较好训练的问题，本文提出一种融合生成对抗网络(Generative Adversarial Network, GAN) 和基于物理的渲染(Physically Based Rendering, PBR) 流程的生成式样本合成方法用于数据增强。该方法以ConSinGAN为缺陷特征扩增模型，并通过引入坐标注意力机制(Coordinate Attention, CA) 来优化鉴别器，使其能更精确识别图像中的缺陷特征。同时调整损失函数，引入重构损失与多尺度结构相似度损失的加权组合以缓解小样本训练中的梯度消失问题并提高生成质量。采用PBR流程输出扩增样本，首先为待扩增样本的工件构建三维模型，然后利用泊松融合将扩增的缺陷特征与原始模型贴图融合，最后在虚拟生产环境中通过虚拟相机渲染输出工件缺陷样本。在公共数据集下的实验结果表明该方法可以对给定的工件缺陷小样本进行有效数据增强。

关键词: 数据增强, 生成对抗网络, 图像生成, 样本合成, 工件缺陷

Abstract: To address the problem of severe lack of defect data in workpieces to train the deep-learning-based defect visual detection systems, this paper introduces a generative sample synthesis method that integrates generative adversarial networks (GAN) with a physical-based rendering (PBR) pipeline for data augmentation. The method employs ConSinGAN as the defect feature generation model and enhances the discriminator by incorporating a coordinate attention (CA) mechanism, enabling more precise identification of defect features in images. Additionally, the loss function is adjusted by introducing a weighted combination of reconstruction loss and multi-scale structural similarity loss to alleviate the gradient vanishing in small sample training and improve the quality of generated samples. The PBR pipeline is used to output the augmented samples, which first constructs a 3D model for the workpiece to be augmented, and then use poisson blending to merge the generated defect features with the original model texture. Finally, defect samples of the workpiece are rendered in a simulated production environment using a virtual camera. Experimental results on public datasets demonstrate the effectiveness of the proposed method in augmenting small samples of workpiece defects.

Key words: data augmentation, generative adversarial network, image generation, sample synthesis, workpiece defect

中图分类号:

TP3-05

李晋芳, 肖立宝, 何明桐, 莫建清. 基于生成式样本合成的工件缺陷样本数据增强[J]. 广东工业大学学报,doi: 10.12052/gdutxb.240056

Li Jinfang, Xiao Libao, He Mingtong, Mo Jianqing. Method for Data Augmentation of Workpiece Defect Samples Based on Generative Sample Synthesis[J]. Journal of Guangdong University of Technology, 2024, 42(0): 0-0.doi: 10.12052/gdutxb.240056

[1] 王慧菁, 杨长辉, 吕庆. 基于机器视觉的金属表面缺陷检测方法综述[J]. 微纳电子与智能制造, 2022, 4(4): 71-81.
WANG H Q, YANG C H, LYU Q. Review of metal surface defect detection methods based on machine vision[J]. Mirco/Nano Electronics and Intelligent Manufacturing, 2022, 4(4): 71-81.
[2] 金映谷, 张涛, 杨亚宁, 等. 基于深度学习的产品缺陷检测方法综述[J]. 大连民族大学学报, 2020, 22(5): 420-427.
JIN Y G, ZHANG T, YANG Y N, et al. Review of product defect detection methods based on deep learning[J]. Journal of Dalian Minzu University, 2020, 22(5): 420-427.
[3] 罗东亮, 蔡雨萱, 杨子豪, 等. 工业缺陷检测深度学习方法综述[J]. 中国科学: 信息科学, 2022, 52(6): 1002-1039.
LUO D L, CAI Y X, YANG Z H, et al. Survey on industrial defect detection with deep learning[J]. Scientia Sinica Informationis, 2022, 52(6): 1002-1039
[4] MUMUNI A, MUMUNI F. Data augmentation: a comprehensive survey of modern approaches[J]. Array, 2022, 16: 100258.
[5] GOODFELLOW I, POPUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139-144.
[6] 孙书魁, 范菁, 孙中强, 等. 基于深度学习的图像数据增强研究综述[J]. 计算机科学, 2024, 51(1): 150-167.
SUN S K, FAN Q, SUN Z Q, et al. Survey of image data augmentation techniques based on deep learning[J]. Computer Science, 2024, 51(1): 150-167.
[7] 庄昌乾, 李璟文. 基于YOLOv5和生成对抗网络的塑料标签缺陷检测[J]. 计算机测量与控制, 2023, 31(7): 91-98.
ZHUANG C Q, LI J W. Industrial defect detection of plastic labels based on YOLOv5 and generative adversarial networks[J]. Computer Measurement & Control, 2023, 31(7): 91-98.
[8] 罗月童, 段昶, 江佩峰, 等. 一种基于pix2pix改进的工业缺陷数据增强方法[J]. 计算机工程与科学, 2022, 44(12): 2206-2212.
LUO Y T, DUAN C, JIANG P F, et al. An improved industrial defect data augmentation method based on pix2pix[J]. Computer Engineering & Science, 2022, 44(12): 2206-2212.
[9] PHILLIP I, ZHU J Y, ZHOU T H, et al. Image-to-image translation with conditional adversarial networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Honolulu: IEEE, 2017: 5967-5976.
[10] LIU J H, WANG C Y, SU H, et al. Multistage GAN for fabric defect detection[J]. IEEE Transactions on Image Processing, 2019, 29: 3388-3400.
[11] 闫艺丹, 孙君顶, 姚冲, 等. 基于生成对抗网络的CT图像数据扩增[J]. 计算机系统应用, 2022, 31(12): 78-86.
YAN Y D, SUN J D, YAO C, et al. CT image data amplification based on generative adversarial network[J]. Computer Systems & Applications, 2022, 31(12): 78-86.
[12] BROCK A, DONAHUE J, SIMONYAN K. Large scale GAN training for high fidelity natural image synthesis[EB/OL]. arXiv: 1809.11096(2019-02-25) [2024-03-16]. https://doi.org/10.48550/arXiv.1809.11096.
[13] KARRAS T, AILA T, LAINE S, et al. Progressive growing of GANs for improved quality, stability, and variation[EB/OL]. arXiv: 1710.10196(2018-02-26) [2024-03-16]. https://doi.org/10.48550/arXiv.1710.10196.
[14] TAMAR R S, TALI D, TOMER M. SinGAN: learning a generative model from a single natural image[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Seoul: IEEE, 2019: 4569-4579.
[15] HINZ T, FISHER M, WANG O, et al. Improved techniques for training single-image gans[C]//2021 IEEE Winter Conference on Applications of Computer Vision (WACV) . Waikoloa: IEEE, 2021: 1299-1308.
[16] 彭晏飞, 邓佳楠, 王刚. 基于改进SinGAN的遥感图像数据增强方法[J]. 液晶与显示, 2023, 38(3): 387-396.
PENG Y F, DENG J N, WANG G. Remote sensing image data enhancement based on improved SinGAN[J]. Chinese Journal of Liquid Crystals and Displays, 2023, 38(3): 387-396.
[17] 赵晓枫, 夏玉婷, 徐叶斌, 等. 地面红外目标数据联合增强方法[J]. 激光与红外, 2023, 53(7): 1117-1124.
ZHAO X F, XIA Y T, XU Y B, et al. Joint data augmentation method for ground infrared target[J]. Laser & Infrared, 2023, 53(7): 1117-1124.
[18] 黄琼男, 朱卫纲, 刘渊, 等. 基于多尺度GAN网络的SAR舰船目标扩充[J]. 兵工自动化, 2022, 41(7): 47-52.
HUANG Q N, ZHU W G, LIU Y, et al. SAR ship target expansion based on multiscale GAN network[J]. Ordnance Industry Automation, 2022, 41(7): 47-52.
[19] ZHANG H, GOODFELLOW I, METAXAS D, et al. Self-attention generative adversarial networks[J]. Proceedings of Machine Learning Research, 2019, 97: 7354-7363.
[20] HU J, SHEN L, SUN G, et al. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[21] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Computer Vision - ECCV 2018: 15th European Conference. Munich: Springer, 2018: 3-19.
[22] HOU Q B, ZHOU D Q, FENG J S. Coordinate attention for efficient mobile network design[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Nashville: IEEE, 2021: 13708-13717.
[23] ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein generative adversarial networks[J]. Proceedings of Machine Learning Research, 2017, 70: 214-223.
[24] GIRSHICK R. Fast R-CNN[C]//2015 IEEE International Conference on Computer Vision (ICCV) . Santiago: IEEE, 2015: 1440-1448.
[25] WANG Z, SIMONCELLI E P, BOVIK A C. Multiscale structural similarity for image quality assessment[C]//The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers. Pacific Grove: IEEE, 2003: 1398-1402.
[26] 姜滟稳, 江自昊. PBR方法在虚拟现实技术中的研究与应用[J]. 黑河学院学报, 2022, 13(6): 178-180.
JIANG Y W, JIANG Z H. On the application of PBR method in virtual reality technology[J]. Journal of Heihe University, 2022, 13(6): 178-180.
[27] PEREZ P, GANGNET M, BLAKE A. Poisson image editing[J]. ACM Transactions on Graphics, 2003, 22(3): 313-318.

Just accepted

Online first

Just accepted

Online first

Viewed

Full text

From	Others	local

Times	20	56
Rate	26%	74%

Abstract

Just accepted	Online first	Issue

0	85	0

	From	local

	Times	85
	Rate	100%

Cited

Web of Science	Crossref	ScienceDirect	Search for Citations in Google Scholar >>


This page requires you have already subscribed to WoS.

Shared

Discussed