Monocular tissue reconstruction via remote center motion for robot-assisted minimally invasive surgery

In minimally invasive surgery, the primary surgeon requires an assistant to hold an endoscope to obtain visual information from the body cavity. However, the two-dimensional images acquired by endoscopy lack depth information. Future automatic robotic surgeries need three-dimensional information of the target area. This paper presents a method to reconstruct a 3D model of soft tissues from image sequences acquired from a robotic camera holder. In this algorithm, a sparse reconstruction module based on the SIFT and SURF features is designed, and a multilevel feature matching strategy is proposed to improve the algorithm efficiency. To recover the realistic effect of the soft-tissue model, a complete 3D reconstruction algorithm is implemented, including densification, meshing of the point cloud and texture mapping reconstruction. During the texture reconstruction stage, a mathematical model is proposed to achieve the repair of texture seams. To verify the feasibility of the proposed method, we use a collaborative manipulator (AUBO i5) with a mounted camera to mimic an assistant surgeon holding an endoscope. To satisfy a pivotal constraint imposed by the remote center of motion (RCM), a kinematic algorithm of the manipulator is implemented, and the primary surgeon is provided with a voice control interface to control the directions of the camera with. We conducted an experiment to show a 3D reconstruction of soft tissue by the proposed method and the manipulator, which indicates that the manipulator works as a robotic assistant which can hold a camera to provide abundant information in the surgery.