A Real-Time 3D Centerline Estimation Framework for Multi-Section Soft Manipulator Based on Stereo Vision

Soft manipulator is a strong nonlinear system with high uncertainty. Real-time 3D shape estimation is the base for the control and application of soft manipulators. However, it is challenging to realize 3D shape estimation through accurate modeling as the rigid manipulator. To deal with this issue, a real-time 3D centerline estimation framework based on stereo vision is proposed for the multi-section soft manipulator in this work. The contour of the manipulator is segmented accurately from the real-time images captured by the ZED camera using the machine vision method. The contour data is then clustered based on the self-organizing mapping (SOM) algorithm to form a 2D centerline. The linear overdetermined equation established by the camera projection model is figured out to obtain the optimal solution in the sense of least squares, and the 3D reconstruction is completed. In the simulation of the SOM algorithm, the parameters selection, simulation verification, and the comparison of various centerline extraction algorithms are completed. The results show that the SOM algorithm has more advantages to solve this work. Real-time bending experiments are carried out to verify the feasibility and robustness of the proposed framework, and performance evaluation experiments are also performed for accuracy and real-time performance.