RES: Reconstruction-based sampling for point cloud learning

Processing large-scale point clouds presents a significant challenge. Recent works address this issue by downsampling the point cloud to reduce its size before further processing. However, they often focus on the overall structure of the point cloud, which leads to the loss of small-scale details. Moreover, they typically rely on generic point cloud encoders without considering the characteristics of the downsampled point clouds. In this paper, we re-examine the design of both downsampling and encoder modules and propose a novel reconstruction-based sampling framework for point cloud learning. Specifically, our sampling strategy consists of two components: Point Reconstruction and Shape Reconstruction, which remove points and then reconstruct them from the remaining ones at different detail levels. We then compute the difference between the reconstructed and original points to measure the salience of each point and achieve feature-preserved downsampling by removing points with lower salience. Finally, to efficiently extract features from point clouds, we propose a Local-Global Feature Aggregation (LGFA) module, which first extracts small-scale details through local attention and then captures the overall structure through global attention. Experiments demonstrate that our method achieves outstanding results across various point cloud analysis and downsampling tasks.