PlanePDM: Boundary-aware 3D planar recovery by using parallel dilated mask head

This paper investigates the problem of recovering 3D planar structures from single RGB images, which aims to segment plane instances and predict their corresponding 3D parameters simultaneously. Despite remarkable progress in this area, current mainstream methods still suffer from two shortcomings: (1) incorrect detection of non-plane regions; (2) unsatisfactory plane restoration quality. To tackle these issues, we first propose utilizing a direct segmentation framework to predict plane instances and their corresponding normal vectors. On this basis, we propose PlanePDM to provide lightweight yet effective boundary supervision for high-quality 3D plane recovery. More specifically, the PlanePDM designs a tailored dilated mask head parallel to the conventional plane mask prediction head. Due to such a design, we can generate boundary predictions of planes by performing simple per-pixel minus operations, thereby avoiding complex post-processing techniques typically required by contour regression methods. Comprehensive experiments demonstrate that PlanePDM outperforms existing state-of-the-art techniques with higher margins in terms of plane detection, segmentation, and reconstruction metrics across the ScanNet and NYUv2 datasets.