Abstract: To address the problem of severe lack of defect data in workpieces to train the deep-learning-based defect visual detection systems, this paper introduces a generative sample synthesis method that integrates generative adversarial networks (GAN) with a physical-based rendering (PBR) pipeline for data augmentation. The method employs ConSinGAN as the defect feature generation model and enhances the discriminator by incorporating a coordinate attention (CA) mechanism, enabling more precise identification of defect features in images. Additionally, the loss function is adjusted by introducing a weighted combination of reconstruction loss and multi-scale structural similarity loss to alleviate the gradient vanishing in small sample training and improve the quality of generated samples. The PBR pipeline is used to output the augmented samples, which first constructs a 3D model for the workpiece to be augmented, and then use poisson blending to merge the generated defect features with the original model texture. Finally, defect samples of the workpiece are rendered in a simulated production environment using a virtual camera. Experimental results on public datasets demonstrate the effectiveness of the proposed method in augmenting small samples of workpiece defects.