Abstract

Even though considerable progress has been made in deep learning-based 3D point cloud processing, how to obtain accurate correspondences for robust registration remains a major challenge because existing hard assignment methods cannot deal with outliers naturally. Alternatively, the soft matching-based methods have been proposed to learn the matching probability rather than hard assignment. However, in this paper, we prove that these methods have an inherent ambiguity causing many deceptive correspondences. To address the above challenges, we propose to learn a partial permutation matching matrix, which does not assign corresponding points to outliers, and implements hard assignment to prevent ambiguity. However, this proposal poses two new problems, i.e. existing hard assignment algorithms can only solve a full rank permutation matrix rather than a partial permutation matrix, and this desired matrix is defined in the discrete space, which is non-differentiable. In response, we design a dedicated soft-to-hard (S2H) matching procedure within the registration pipeline consisting of two steps: solving the soft matching matrix (S-step) and projecting this soft matrix to the partial permutation matrix (H-step). Specifically, we augment the profit matrix before the hard assignment to solve an augmented permutation matrix, which is cropped to achieve the final partial permutation matrix. Moreover, to guarantee end-to-end learning, we supervise the learned partial permutation matrix but propagate the gradient to the soft matrix instead. Our S2H matching procedure can be easily integrated with existing registration frameworks, which has been verified in representative frameworks including DCP, RPMNet, and DGR. Extensive experiments have validated our method, which creates a new state-of-the-art performance.

Contribution

• We theoretically analyze the inherent ambiguity in the soft matching-based methods, which causes serious de- generation of the learned virtual corresponding points.
• WeproposeanovelS2Hmatchingproceduretolearnthe PPM, which handles the outliers and prevents ambiguity synchronously. This matching procedure not only solves a real PPM, but also guarantees end-to-end learning.
• Remarkable performance on benchmark datasets verifies the proposed method, which achieves state-of-the-art performance in robust 3D point cloud registration.

Motivation

Comparison of learning-based point cloud registration methods based on “soft” matching and “hard” matching. Points with different colors indicate the source (green), target (blue), and virtual points (red). $m_{ij}$ is the entry of matching matrix $\mathbf{M} \in \mathbb{R}^{N_\mathcal{X} \times N_\mathcal{Y}}$, where $N_\mathcal{X}$, $N_\mathcal{X}$ are size of the source and target. (The matching result of ours is returned by SHMDCP.)

Network Architecture

S2H matching procedure in registration pipeline. Given the source and target, the similarity matrix is obtained based on the point features. Then, S2H is applied for the final PPM. Finally, the rigid motion is estimated by the weighted Procrustes.

Result Visualization

For each pair of point clouds, left upper is the transformed source using the predicted rigid motion and right bottom is the target. The same color represents the cor- respondences, and black indicates the abandoned points.