Aggregated Data Augmentation for Defects Using Enhanced Diffusion Models and Poisson Blending

Detecting defects of workpiece surface is crucial for maintaining operational reliability and safety in the industry. However, obtaining defect samples of workpiece such as cracks, pinholes, and folds, is labor-intensive and time-consuming. Moreover, the samples collected often exhibit class imbalance, leading models to easily fall into local optima. Currently methods, such as under-sampling and over-sampling, all rely on a limited number of real samples and suffer from a lack of data diversity. To address these chanlledges, this paper proposes the aggregated data augmentation for defects using enhanced diffusion models and poisson blending. For defects such as cracks and pinholes, we propose a Segmentation Driven Generation method combining Segmenting Everything In Context(Seggpt) segmentation with Open-Set Grounded Text-to-Image Generation(GLIGEN) prompt generation to create defect samples for corresponding categories. For more complex defects such as folds, we introduce a module named PoissonSeg that integrates Seggpt segmentation with the Poisson equation to generate samples of complex defects. Taking the steering wheel defect data set as an example, experiments results, including the comparisons and single-category ablation studies demonstrate using our proposed method significantly enhances the detection performance of six different defect detection algorithms on steering wheel defects, with increases of 0.144 in terms of mAP0.5, proving the effectiveness of the proposed method in data augmentation for defect detection.