A Simulation-based Pipeline for Data Generation and Validation in Learning-Based Visual Servoing

Visual servoing is a critical technique in robotic applications, enabling robots to accurately reach target positions using visual information. Traditional methods, while offering high precision, often suffer from limited convergence domain and strong reliance on prior knowledge. In contrast, learning-based approaches alleviate these limitations but typically face challenges in generalization due to limited training data. Recent studies have demonstrated that leveraging large-scale synthetically generated data in simulation can significantly improve the generalization ability of learning-based models. Inspired by these works, this paper presents an end-to-end pipeline for data generation, model training, and simulation deployment based on Isaac Sim. Leveraging its high-fidelity visual simulation capabilities, the proposed framework enables large-scale dataset generation and algorithm validation for visual servoing tasks. Experimental results demonstrate that models trained with the generated large-scale datasets exhibit improved generalization (over 13% increase of success rate) and faster convergence (approximately 2.5× faster) under specific supervision settings. This paper confirms the effectiveness of large-scale data in enhancing model performance and offers a complete and efficient solution for algorithm development and evaluation in visual servoing.